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PREFACE 
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Training,  and  1 123-B2-06,  Aircrew  Training  Research  Support.  The  effort  was  sponsored  by  the 
Air  Force  Material  Command,  Human  Systems  Center’s  Training  Technology  Planning 
Integrated  Planning  Team  (Training  TPIPT)  directed  by  Lt  Col  Peter  Demitry  and  Capt  Christine 
Poprik.  The  effort  was  supported  by  Hughes  Training,  Inc.  (HTI),  under  Contract  F41624-95-C- 
501 1 .  Principal  Investigator  for  Armstrong  Laboratory's  Aircrew  Training  Research  Division 
(AL/HRA)  was  Dr  Peter  M.  Crane;  Laboratory  Contract  Monitor  was  Mr  Daniel  H.  Mudd. 
Portions  of  this  research  were  conducted  in  cooperation  with  the  National  Aeronautics  and  Space 
Administration’s  Dry  den  Flight  Research  Center  (NASA  DRFC)  and  the  University  of  Central 
Florida.  We  would  like  to  thank  Rogers  Smith  and  Larry  Schilling  from  NASA  DFRC;  Vem 
Carter  from  Northrop-Grumman  Corp.;  Matt  Schifter,  Pedro  Claudio,  Dale  Jewell,  Chuck  Lutz, 
Matt  Archer,  Laura  Miller,  Soni  Basi,  and  Bud  Conyers  of  ECC.  Our  special  thanks  to  Jack  Kolf 
from  NASA  DFRC  who  originated  the  concept  of  above  real-time  training  (i.e.,  “fast-time 
simulation”)  while  working  on  the  NASA  X- 15  and  Lifting  Body  research  programs  during  the 
1960s  and  70s. 

Special  thanks  go  to  the  pilots  of  the  USAF  68th  Fighter  Squadron  (FS),  69th  FS,  and  the 
347th  Operations  Support  Squadron  from  the  347th  Fighter  Wing  at  Moody  Air  Force  Base, 
Georgia;  and  the  61st  FS,  63rd  FS,  308th  FS,  and  309th  FS  from  the  58th  Fighter  Wing,  Luke  Air 
Force  Base,  Arizona. 


ABOVE  REAL-TIME  TRAINING  APPLIED  TO  AIR  COMBAT  SKILLS 

1.  INTRODUCTION 

Design  of  flight  training  simulators  is  often  based  on  the  concept  that,  "transfer  of  training  is 
highest  when  similarity  of  the  training  and  transfer  situations  is  the  highest  fidelity  . . .  this  is  the 
governing  principle  for  most  simulators  that  are  built,"  (Adams,  1979,  p.  717).  Although  there 
have  been  many  efforts  directed  toward  discovering  when  high  fidelity  simulation  is  not  required 
for  effective  training  (see  Adams  1989,  ch  18),  there  have  been  almost  no  attempts  to  deliberately 
reduce  simulator  fidelity  in  order  to  increase  training  effectiveness.  In  the  early  1970s,  however, 
engineers  and  test  pilots  at  the  National  Aeronautics  and  Space  Administration’s  (NASA) 

Dryden  Flight  Research  Center  apparently  increased  the  effectiveness  of  flight  training 
simulators  by  deliberately  distorting  simulated  time. 

1.1  NASA's  "Fast-Time"  Simulations 

Kolf  (1973,  Appendix  A)  and  Hoey  (1976,  Appendix  B)  briefly  document  simulator  training 
interventions  which  were  aimed  at  improving  test  pilots'  ability  to  keep  up  with  the  pace  of 
events  in  flight.  Kolf  notes  that,  "regardless  of  the  type  or  amount  of  pre-flight  simulator  training 
accomplished  by  the  pilot,  the  actual  flight  seems  to  take  place  in  a  much  faster  time  frame  than 
real  time,"  (Appendix  A,  p.  1).  Hoey  (1976)  reports  that  in  the  X-15  program,  pilots  typically 
spent  ten  hours  in  the  simulator  for  each  ten  minutes  of  flight.  Even  with  this  preparation,  pilots 
reported  that,  "It  sure  seems  to  happen  faster  in  the  real  airplane,"  or,  "I  had  the  feeling  that  I  was 
'behind  the  airplane' ",  (Appendix  B,  pp.  2  -  3). 

As  an  experiment,  Kolf  increased  the  rate  of  simulated  time  in  the  M2-F3  Lifting  Body 
simulator.  The  effect  of  this  "fast-time"  modification  was  to  increase  the  pace  of  events  by 
changing  the  simulator’s  time  integration  factor.  In  the  modified  simulator,  a  mission  profile 
which  normally  required  10  minutes  to  complete  took  place  in  only  6  minutes,  40  seconds. 
Unfortunately,  the  Lifting  Body  test  program  was  canceled  and  there  were  no  opportunities  for  a 
pilot  to  fly  a  mission  profile  in  fast-time  before  flying  the  actual  mission.  However,  three 
experienced  M2-F3  pilots  flew  a  familiar  mission  at  1.5  times  real  time  and  all  agreed  with 
"enthusiastic  responses,"  (Appendix  A,  p.  2)  that  the  modified  simulator  felt  exactly  like  the 
aircraft. 

Hoey  (1976)  describes  further  applications  of  fast-time  simulation  at  NASA.  The  response  of 
the  technical  community  to  Kolf  s  M2-F3  experiment  had  been  that  the  physical  environment  of 
flight  is  so  different  from  simulation  that  it  was  impossible  to  draw  inferences  about  training  and 
fast-time.  The  next  application  of  fast-time,  however,  was  to  a  flight  test  program  for  remotely 
piloted  vehicles  (RPV).  In  this  case,  the  training  environment  was  exactly  the  same  as  the  actual 
flight  environment.  Nevertheless,  RPV  pilots  who  used  simulation  at  1.4  times  real  time  as  final 
preparation  before  a  flight  (Appendix  B,  pp.  5  and  18)  reported  being,  "Less  rushed  and  more 
confident,"  (p.l  8)  than  when  using  real-time  training  exclusively.  A  typical  practice  was  to 
conduct  70%  of  training  at  real  time  with  the  last  30%  at  1 .4  times  real  time.  Hoey  reports, 
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however,  that  information  about  these  experiments  with  fast-time  simulation  was  not  widely 
disseminated  and  no  further  research  was  conducted. 

1.2  Temporal  Plasticity 

While  there  has  been  little  research  directly  relating  apparent  time  to  training,  it  is  well 
documented  that  human  perception  of  time  is  highly  adaptable  depending  on  circumstances. 
Anticipating  a  pleasant  experience  (Edmonds,  Cahoon,  &  Bridges,  1981),  boredom  (DeWolfe  & 
Duncan,  1959),  or  literally  waiting  for  a  pot  to  boil  (Cahoon  &  Edmonds,  1980)  can  increase  the 
perceived  duration  of  time.  Similarly,  adaptation  to  rapidly  paced  events  seems  to  decrease  the 
perceived  duration  of  time.  Mathews  (1978)  reports  that  drivers  adapted  to  freeway  speeds  travel 
on  city  streets  7%  faster  than  non-adapted  drivers.  Casey  &  Lund  (1987)  report  that  this  effect 
occurs  even  if  freeway  drivers  are  required  to  stop  before  entering  city  streets. 

1.3  Applications  of  Temporal  Plasticity 

The  research  on  temporal  plasticity  cited  above  has  concerned  the  effects  of  different  factors 
on  time  perception  as  a  dependent  variable.  Other  researchers  have  used  apparent  time  as  an 
independent  variable.  LaBarbera  &  MacLachan  (1979)  used  a  compressor-expander  to  increase 
the  presentation  rate  for  tape  recorded  speech  without  distorting  pitch.  Subjects  listened  to  radio 
commercials  presented  in  normal  or  fast  mode  and  reported  greater  interest  and  higher  brand 
name  recall  for  fast  mode  messages.  McLachan  &  LaBarbera  (1978)  found  increased  ratings  of 
reported  interest  and  increased  brand-name  recall  for  time-compressed  television  advertisements. 

1.4  Time  and  Performance  in  Training 

The  NASA  efforts  reported  by  Kolf  (1973)  and  by  Hoey  (1976)  used  time  as  an  independent 
variable.  More  typically,  research  on  training  uses  the  time  required  to  complete  a  task  as  a 
dependent  measure  of  skill  acquisition.  Fitts  &  Posner’s  (1967)  model  of  skill  acquisition  uses 
tune  and  errors  to  define  three  stages  in  learning  a  skill.  In  the  first  or  cognitive  stage,  the  trainee 
is  given  instruction  in  facts  and  procedures.  Performance  is  slow  and  errors  are  frequent.  The 
trainee  must  rely  on  declarative  knowledge  to  perform  the  task.  In  the  second  or  associative 
stage,  declarative  knowledge  is  transformed  into  procedural  knowledge  and  errors  in 
understanding  are  eliminated.  Time  required  for  performance  is  reduced  but  performance  can  be 
disrupted  by  unexpected  events.  In  the  third  or  autonomous  stage,  performance  is  most  rapid  and 
error-free.  In  this  stage,  performance  is  entirely  based  on  procedural  knowledge  and  the 
performer  may  not  be  able  to  verbalize  the  declarative  knowledge  used  to  solve  a  problem. 

The  associative  and  autonomous  phases  in  Fitts  and  Posner’s  model  correspond  well  with  the 
phenomenon  of  controlled  vs.  automatic  processing  (see  James  1890,  Schneider  1985). 

"Automatic  processing  is  a  fast,  parallel,  fairly  effortless  process  that  is  not 
limited  by  short-term  memory  capacity,  is  not  under  direct  subject  control,  and 
performs  well-developed,  skilled  behaviors.  Automatic  processing  typically 
develops  when  subjects  deal  with  the  stimulus  consistently  over  many  trials. 
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Controlled  processing  is  characterized  as  a  slow,  generally  serial,  effortful, 
capacity-limited,  subject-controlled  processing  mode  that  must  be  used  to  deal 
with  novel  or  inconsistent  information."  (Schneider,  1985,  pp.  296-297) 

Controlled  processing  is  characteristic  of  a  trainee  in  the  associative  stage  while  the  autonomous 
stage  demonstrates  automatic  processing.  Klein’s  (1989)  model  of  recognition-primed  decisions 
and  Ericsson  &  Poisson’s  (1988)  analysis  of  skilled-memory  illustrate  the  differences  in 
performance  between  controlled  and  automatic  processing. 

Hoey  (1976)  conducted  interviews  with  test  pilots  who  had  experienced  above  real-time 
training  (ARTT)  when  participating  in  the  RPV  program.  These  individuals  were  highly 
experienced  pilots  with  several  thousand  flight  hours.  Automatic  processing  for  flight  skills  was 
not  an  issue.  However,  to  maximize  data  return  from  a  given  test  flight,  mission  profiles  were 
designed  to  be  demanding  and  to  increase  pilot  workload  on  successive  flights.  The  pilots 
reported  that  using  ARTT  as  the  final  step  in  mission  preparation  helped  them  feel  more 
confident,  less  rushed,  and  better  able  to  handle  the  workload.  They  also  thought  that  ARTT 
should  be  applied  to  training  high  workload  tasks  such  as  instrument  approaches  or  emergency 
procedures  and  that  it  could  be  used  to  evaluate  task  planning  effectiveness  and  pilot  readiness. 
Overall,  the  NASA  experience  suggests  that  ARTT  as  a  training  strategy  provides  benefits  that 
result  from  automatic  processing:  reduced  workload  and  less  time  required  to  complete  tasks. 

1.5  Recent  Research  on  ARTT 

The  NASA  application  of  ARTT  was  limited  to  experts  preparing  for  specific  missions. 
Manipulating  apparent  time  has  been  evaluated  more  recently  as  an  instructional  tool  using  both 
novices  and  experienced  individuals  as  trainees. 

1 .5.1  ARTT  applied  to  training  novices.  Schneider,  Vidulich,  &  Yeh  (1982)  and,  Vidulich, 
Yeh,  &  Schneider  (1983)  used  time-compression  to  help  train  air  traffic  controllers.  The  task 
was  for  the  controller  to  monitor  an  aircraft’s  flight  path  on  a  radar  display  and  issue  turning 
instructions  so  that  the  aircraft  would  fly  through  a  specific  vector.  Actual  aircraft  would 
traverse  20  nautical  miles  and  require  approximately  five  minutes  at  260  knots  to  complete  the 
turn.  These  researchers  increased  the  apparent  rate  of  time  in  the  simulator  to  20  times  real  time 
so  that  a  turn  would  be  complete  in  approximately  1 5  seconds.  The  primary  effect  of  this  high 
level  of  time  compression  was  to  make  it  easier  for  the  novice  controller  to  see  the  aircraft’s  turn. 
Secondary  effects  were  that  the  rapid  pace  of  events  focused  the  trainee  controller’s  attention  to 
the  immediate  task  and  to  allow  more  training  trials  per  hour  than  real-time  simulation. 

Vidulich,  Yeh,  &  Schneider  (1983)  trained  university  students  over  four  hours  to  perform  a  turn 
point  task.  A  group  of  students  who  performed  the  task  in  real  time  using  a  high-fidelity 
simulation  of  an  aircraft  traveling  at  260  knots  experienced  approximately  32  trials  in  four  hours 
of  training.  A  group  performing  the  same  task  using  ARTT  at  20  times  real-time  received 
approximately  260  time-compressed  trials  followed  by  only  3  or  4  real-time  trials  in  four  hours 
of  training.  All  trainees  were  tested  at  real  time  for  two  hours.  ARTT  subjects  showed 
significantly  better  performance  at  initiating  turns  properly.  These  authors  assert  that  the  ARTT 
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training  procedure  encourages  automatic  processing  by  allowing  many  trials  and  training  under  a 
mild  speed  stress. 

Guckenberger,  Uliano,  &  Lane  (1992)  trained  novices,  university  students,  in  tank  gunnery 
using  several  ARTT  conditions.  In  this  experiment,  students  were  trained  in  gunnery  tasks 
which  required  them  to  detect,  identify,  and  shoot  a  moving  target  using  an  Ml  tank  part-task 
trainer.  Students  received  five  familiarization  trials  in  real  time  followed  by  15  training  trials  in 
real  time  or  in  one  of  four  ARTT  conditions.  Students  were  then  tested  in  real  time.  Subjects  in 
all  four  ARTT  groups  showed  better  performance  on  test  trials  than  the  students  trained  in  real 
time. 

Schneider  (1989)  proposes  that  the  primary  effect  of  time  compression  is  to  allow  more 
training  trials  within  a  given  period  of  clock  time.  In  the  air  traffic  control  studies,  subjects  were 
given  the  same  amount  of  training  time  in  the  simulator  so  that  the  ARTT  subjects  received  more 
training  trials.  In  contrast,  Guckenberger  et  al.  gave  all  subjects  the  same  number  of  training 
trials  so  that  the  ARTT  subjects  received  less  training  time  than  the  students  trained  in  real  time. 
Since  the  students  trained  using  ARTT  performed  better  on  real-time  test  trials  than  students 
trained  in  real  time,  Guckenberger  et  al.’s  results  indicate  that  ARTT  has  a  beneficial  effect 
beyond  simply  increasing  the  number  of  training  events.  It  should  be  noted,  however,  that  the 
Guckenberger  et  al.  findings  are  based  on  a  task  familiar  to  anyone  who  has  played  video  games, 
i.e.,  locate  the  target,  put  the  cross-hairs  on  the  target,  and  shoot.  In  a  different  study, 
Guckenberger,  Guckenberger,  Stanney,  &  Mapes  (1993)  used  a  very  unfamiliar  task  to  compare 
the  effects  of  normal  time  vs.  fast-time  training.  In  this  experiment,  university  students  wore  a 
virtual-reality,  helmet-mounted  display  and  a  gesture  recognition  glove.  Subjects  used  the 
helmet-mounted  display  to  watch  cubes  moving  in  three-dimensional  space.  They  would  also 
see  a  representation  of  their  hand  which  was  wearing  the  glove.  Subjects  were  instructed  to 
touch  the  hand  icon  to  the  cube  as  quickly  as  possible.  Subjects  were  trained  at  either  a  standard 
rate  of  motion  or  at  1 .7  times  the  standard  rate.  All  subjects  then  tested  at  the  standard  rate. 
Guckenberger  et  al.  found  no  evidence  for  improved  performance  for  the  ARTT  compared  to 
standard  time  subjects,  however,  the  perceived  workload  of  the  ARTT  group  was  significantly 
less  than  that  of  the  standard  time  group  during  the  testing  phase. 

1 .5.2  ARTT  applied  to  training  for  experienced  individuals.  Guckenberger,  Uliano,  Lane, 

&  Stanney  (1992)  conducted  an  experiment  using  24  experienced  F-16C  pilots.  The  pilots  used 
a  low-cost,  F-16A  trainer  to  fly  several  different  missions.  Pilots  trained  at  real-time,  1.5  times 
real  time,  2.0  times  real-time,  or  with  a  random  mix  of  apparent  times.  Pilots  then  tested  at  real 
time.  The  results  for  1  vs.  2  air  combat  maneuvering  and  a  stem  conversion  task  were 
inconclusive  due  to  difficulties  in  scoring  the  data.  A  third  task  required  the  pilot  to  engage  a 
bandit  and,  to  perform  a  complex  albeit  contrived  threat  response  when  a  warning  was  detected. 
For  this  third,  dual-threat  task,  the  2.0  times  real-time  and  mixed  ARTT  groups  showed  faster 
threat  response  than  the  group  trained  in  real-time  and,  all  ARTT  groups  achieved  significantly 
more  bandit  kills  during  real-time,  test  trials. 
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1 .6  Summary  of  Research  on  ARTT 

The  reported  literature  on  ARTT  is  promising  but  not  conclusive.  The  NASA  experience 
with  ARTT  strongly  suggests  that  time-compressed  simulation  can  be  used  to  augment 
conventional  simulator  training  in  order  to  encourage  automaticity  and  reduce  pilot  workload. 
Following  Fitts  &  Posner’s  (1977)  three  stage  model  of  skill  acquisition,  NASA’s  application  of 
ARTT  suggests  that  ARTT  should  be  more  useful  for  the  second  or  associative  stage  of  training 
rather  than  for  the  first  or  cognitive  stage.  There  is  limited  experimental  evidence  to  support  this 
position.  Schneider,  Vidulich  &  Yeh  (1982)  and,  Vidulich,  Yeh,  &  Schneider  (1983)  used 
ARTT  with  great  success  with  novices  as  trainees.  These  studies  used  an  extreme  level  of 
ARTT,  20  times  real  time,  in  order  to  time  compress  events  that  take  too  long  to  see  in  real-time. 
The  novice  subjects  in  these  studies  received  extensive  training  on  the  task  suggesting  that  for 
much  of  their  training  they  were  in  the  associative  rather  than  the  cognitive  stage.  Guckenberger, 
Uliano,  &  Lane  (1992)  also  trained  novices  using  ARTT.  In  a  tank  gunnery  task,  Guckenberger 
et  al.  found  that  ARTT  increased  performance  on  a  real-time  transfer  task.  Guckenberger, 
Guckenberger,  Stanney,  &  Mapes  (1993)  found  that  ARTT  provided  decreased  perceived 
workload  but  no  significant  training  benefits  for  novice  subjects  learning  an  unfamiliar  task. 
Finally,  Guckenberger,  Uliano,  Lane,  &  Stanney  (1992)  found  ARTT  to  be  beneficial  for  a  group 
of  experienced  pilots  in  a  dual-threat  task.  A  shortcoming  of  the  Guckenberger  et  al.  (1992) 
study  was  that  the  simulator  was  of  such  limited  capability  that  threat  response  training  tasks 
were  overly  contrived. 

Overall,  previous  research  suggests  that  ARTT  should  be  an  effective  training  strategy  for: 

a)  Tasks  which  require  significant  effort  at  time  and  workload  management,  and 

b)  Trainees  who  have  completed  the  cognitive  portions  of  skill  acquisition. 

For  this  combination  of  task  and  trainee,  ARTT  represents  an  instructional  tool  consistent  with 
Schneider’s  (1985)  guidelines  for  training  complex  skills,  i.e.: 

•  Design  training  to  allow  many  trials  of  critical  skills. 

•  Maintain  active  participation  by  minimizing  passive  observation  of  the  task. 

•  Train  under  mild  speed  stress. 

•  Train  strategies  that  minimize  operator  workload. 

•  Train  time-sharing  skills  for  dealing  with  high-workload  environments,  (pp.  297-298) 

If  used  inappropriately,  ARTT  may  also  violate  Schneider’s  rule  that  training  not  overload 
temporary  (i.e.,  working)  memory.  Limiting  ARTT  to  trainees  who  have  completed  the 
cognitive  stage  of  skill  acquisition  should  minimize  this  problem.  It  is  also  possible  that  a  time- 
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compressed  simulation  will  be  noticeably  different  from  the  real  world  and  lead  to  negative 
transfer  of  training. 

1 .7  Potential  Application  of  ARTT  in  Air  Force  Training 

The  Air  Force  Material  Command’s  Human  Systems  Center  (HSC)  has  established  a 
Technical  Planning  Integrated  Product  Team  (TPIPT)  for  training  to  investigate  technologies 
such  as  ARTT.  The  TPIPT  has  proposed  that  ARTT  may  benefit  Air  Force  training  by: 

1 .  Increasing  task  performance. 

2.  Increasing  trainee  retention  of  skills. 

3.  Increased  situation  awareness. 

4.  Decreasing  real-time  workload. 

5.  Decreasing  real-time  stress. 

6.  Increasing  the  rate  of  skill  acquisition  (faster  and  steeper  learning  curves). 

7.  Reduced  simulator  and  aircraft  training  time. 

Research  has  been  conducted  to  assess  whether  ARTT  can  produce  the  benefits  proposed  by 
HSC’s  Training  TPIPT.  The  present  research  effort  focused  on  air  radar  interpretation/air 
intercept  and  emergency  procedures.  Previous  research  on  ARTT  has  used  university  students  as 
trainees  and/or  low  fidelity  simulators.  The  research  described  in  this  report  has  employed  Air 
Force  F-16  pilots  and  student  pilots,  high-fidelity  simulators,  and  training  problems  which 
emphasize  skills  required  for  air  combat. 

1.8  Report  Organization 

Following  this  introduction,  the  report  contains  four  major  sections: 

•  Engineering  modifications  required  for  changing  apparent  time  in  a  real-time  simulator, 

•  Experiment  1 :  ARTT  for  air  combat  skills  with  experienced  pilots, 

•  Experiment  2:  ARTT  for  emergency  procedures  training  with  experienced  pilots,  and 

•  Experiment  3:  ARTT  for  radar-skills  training  with  student  pilots. 

The  report  concludes  with  a  summary  and  discussion  plus  recommendations  for  further  research. 
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2.  ENGINEERING  FOR  ARTT 


Modifying  real-time  simulators  for  above  real-time  operations  has  not  proven  to  be  difficult 
or  costly.  The  major  problem  has  been  to  overcome  misunderstandings  regarding  the  concept  of 
ARTT. 

2.1  Definitions 

•  Clock  time  and  Real  time  refer  to  the  unalterable  real-world  passage  of  time. 

•  Simulator  time  is  the  apparent  rate  of  time  in  a  simulated  environment.  Most  often, 
simulator  time  equals  real  time. 

•  Time  integration  factor  or  deltat  refers  to  the  amount  of  simulator  time  the  simulator 
assigns  to  have  passed  between  hardware  ticks.  This  assignment  is  usually  a  hard-coded  value 
that  the  simulation  uses  for  all  timing  functions.  For  example,  in  a  real-time  simulator  operating 
at  60  Hz,  the  time  integration  factor  [delta_t]  assigned  between  two  hardware  ticks  would  be 
l/60th  of  a  second  or  16.6  milliseconds. 

•  Less  than  real  time  refers  to  simulator  time  passing  slower  than  real  time.  A  slow  motion 
movie  demonstrates  less  than  real  time  in  that  movie  presentation  time  is  slower  than  real  time. 

•  Above  real  time,  fast-time,  or  time  compression  refers  to  simulator  time  passing  faster  than 
real  time.  Time-lapse  photography  showing  a  flower’s  growth  for  a  month  in  one  minute 
demonstrates  above  real  time. 

•  Hyper-time  algorithm  is  a  procedure  for  alteration  of  time  in  a  real-time  simulator.  The 
hyper-time  algorithm  provides  an  interface  which  allows  dynamic  manipulation  of  the  time 
integration  factor  so  that  a  simulator  can  be  made  to  operate  at  less  than  real-time  or  above  real¬ 
time.  The  hyper-time  factor  is  a  scalar  value  which  describes  the  relationship  between  real  time 
and  simulated  time.  A  hyper-time  factor  less  than  one  results  in  less  than  real-time  training  while 
a  factor  greater  than  one  produces  above  real-time  simulation. 

2.2  Effects  of  changing  simulator  time. 

When  describing  their  air  traffic  control  task,  Vidulich,  Yeh,  and  Schneider  (1983) 
characterize  simulation  at  20  times  real  time  as,  “simulating  an  aircraft  flying  at  5200  knots,”  (p. 
162).  While  this  characterization  is  easy  to  understand,  it  leads  to  misunderstanding.  A  fighter 
aircraft  simulator  operating  at  1 .5  times  real  time  might  be  described  as  having  the  aircraft  fly  at 
450  knots  rather  than  at  300  knots.  This  description  is  incorrect.  An  aircraft  flying  at  450  knots 
has  different  handling  characteristics  such  as  turn  radius  than  the  same  aircraft  at  300  knots. 

Also,  other  systems  on-board  the  aircraft  such  as  radar  and  other  entities  in  the  simulation  have 
not  changed  their  simulated  time.  More  correctly,  in  a  simulation  operating  at  1.5  times  real 
time,  everything  happens  1 .5  times  faster.  Ownship  moves  over  the  earth  faster,  the  radar 
antenna  sweeps  faster,  enemy  aircraft  move  faster,  and  missiles  fly  faster. 


7 


While  time  compression  seems  to  benefit  training,  it  will  lead  to  simulator  effects  being 
physically  incorrect.  If  a  simulated  aircraft  flies  at  300  knots  and  a  bank  angle  of  30°,  its  turn 
rate  is  approximately  126°  per  minute.  When  increasing  simulated  time  to  2.0  times  real  time, 
indicated  speed  and  turn  rate  are  unaffected  while  in  the  physical  world;  doubling  air  speed 
should  decrease  turn  rate  by  half  for  a  given  bank  angle.  In  ARTT  at  2.0  times  real  time,  the  out- 
the-window  appearance  will  be  that  of  flying  at  600  knots  but  with  a  turn  rate  of 252°  per  minute 
of  clock  time.  Such  anomalies  have  the  potential  for  negative  transfer  of  training  from  ARTT  to 
real-time  flight. 

2.3  Inappropriate  implementation  of  ARTT 

A  technique  often  suggested  for  implementing  ARTT  is  to  instruct  the  pilot  to  fly  faster. 
While  this  will  decrease  the  amount  of  time  available  to  perform  a  task,  it  will  also  change  the 
task.  As  described  above,  aircraft  behave  differently  at  different  speeds  and  having  a  pilot  fly  a 
mission  segment  at  600  knots  rather  than  at  a  normal  400  knots  changes  the  task  leading  to  the 
potential  for  negative  transfer  of  training.  Further,  the  timing  events  other  than  ownship  position 
such  as  radar  antenna  sweep  rate  and  missile  fly-outs  will  be  unaffected. 

2.4  Time-Warping  algorithms. 

Two  basic  types  of  algorithms  changes  have  been  utilized  to  implement  ARTT.  The  first 
method  is  to  change  the  basic  update  rate  of  the  system.  If  a  simulator  system  normally  operates 
at  60  Hz,  increasing  its  update  rate  to  90  Hz  produces  ARTT  at  1 .5  times  real  time.  Modifying  a 
simulator  system  to  change  its  update  rate  is  a  viable  technique.  Update  rate  alteration  is  the 
technique  preferred  at  NASA  Dryden  Flight  Research.  Conversion  of  the  time  base  by  alteration 
of  the  update  rate  requires  no  alteration  to  simulation  model  software  and  can  be  implemented 
without  costly  changes.  As  a  simple  illustration,  consider  the  case  where  a  simulator  responds  to 
hardware  interrupts  30  milliseconds  apart.  By  altering  the  hardware  interrupts  to  occur  every 
22.5  milliseconds,  we  have  produced  1.5X  simulation  time.  Modification  using  this  technique 
usually  requires  providing  a  change  to  the  hardware  clock  and  analysis  to  confirm  sufficient 
spare  processing  time  to  satisfy  the  predicted  update  rate.  This  method  is  frequently  cost 
effective  and  requires  little  change  to  the  software. 

The  second  method  is  referred  to  as  the  Hyper-Time  Algorithm,  and  has  been  primarily 
utilized  in  converting  existing  simulation  systems.  These  systems  are  modified  by  altering  the 
software  encoded  time  integration  factor.  Specifically,  the  hyper-time  algorithm  is  a  technique 
for  implementing  ARTT  by  altering  the  amount  of  assigned  simulated  time  between  frame 
updates.  The  key  to  understanding  how  the  hyper-time  algorithm  functions  is  understanding  that 
the  assigned,  simulated  time  is  an  alterable  value  which  the  simulation  model  uses  for  its 
ownship  calculations.  When  the  hyper-time  algorithm  is  operating,  the  simulation  model 
produces  the  same  number  of  frame  updates  as  during  real-time  operation  but  more  or  less 
simulated  time  has  passed  between  updates.  In  a  system  operating  at  real  time,  the  time 
integration  factor,  delta_t,  equals  the  time  between  hardware  updates.  As  an  example,  assume 
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that  a  system  operates  at  100Hz  with  delta_t  equal  to  10  msec  (.01  s).  By  multiplying  delta  t  by 
2,  the  simulation  will  calculate  the  next  update  as  if  twice  (20  ms)  as  much  time  has  passed  as  the 
real-time  simulation  (2X  above  real  time).  If  delta_t  is  multiplied  by  0.25,  the  simulation  will 
calculate  the  next  update  as  if  one  fourth  (2.5  ms)  as  much  time  has  passed  as  the  original  (0.25X 
less  than  real  time).  Update  rate  or  frame  time  is  unaffected.  A  30  Hz  simulator  still  computes 
30  updates  per  second.  When  operating  in  real  time,  each  of  the  30  frames  are  computed  as  if  33 
ms  has  passed.  In  3  times  above  real  time,  each  frame  is  computed  as  if  99  ms  have  passed. 

From  the  pilots’  view,  aircraft  behavior,  out-the-window  imagery  and,  instruments  are  all 
synchronized  to  the  simulated  time.  For  simulations  running  at  a  non-deterministic  frame  rate, 
the  time  constant,  delta_t,  is  calculated  between  frames  by  calling  an  operating  system  time 
function  each  interation  and  computing  the  difference  between  successive  times  to  determine  the 
current  frame  time  (Guckenberger  et  al.,  1995). 

2.4. 1  Sample  implementation  of  the  hyper-time  algorithm.  The  following  example 
describes  a  procedure  for  modifying  the  Silicon  Graphics,  Inc.  (SGI)  Flight  software  for  above  or 
less  than  real  time  operation.  (Flight  is  a  free  demonstration  included  with  all  SGI  products.) 

Step  1 .  Identify  the  frame  rate  control  function(s).  Identify  the  Executive  or  portion  of  code 
that  controls  simulated  timing  of  the  application.  In  the  case  of  SGI  s  Flight,  search  the  source 
code  for  “CLOCK.” 

Step  2.  Define  a  variable  called  HTA_F  ACTOR  in  the  declaration  portion  of  the  program. 
Ensure  that  the  HTA_F  ACTOR  is  of  the  same  type  as  the  time  integration  factor  to  avoid  round¬ 
off  or  truncation  errors. 

Step  3.  Modify  the  simulated  time  number,  “CLOCK”,  by  multiplying  by  the 
HTAF  ACTOR.  SGI’s  Flight  example: 

Existing  CLOCK_RATE  is  altered  by  multiplying  by  HTA,  that  is, 

original  CLOCKRATE  =  40; 

modified  CLOCK  RATE  =  40  *  HTA_F ACTOR 

(Note:  Remove  CONSTANT  from  the  header  definition  of  CLOCK  RATE  to  allow  dynamic 
reassignment  of  the  variable  value) 

Step  4.  Alter  user  interface  to  allow  the  user  to  input  desired  HTAF  ACTOR.  Example: 
printf("\n  Enter  the  desired  HTAF  ACTOR  (  float  format)  \n"); 
scanf("%f ’,  HTA_F  ACTOR); 

printf("\n  You  Entered  HTA_F ACTOR  =  ",  HTA_FACTOR ); 

Step  5.  (Optional)  Put  error  checking  code  to  prevent  inadvertent  boundary  user  errors 
Example: 

if  ((HTA_F ACTOR  >  10)  ||  (HTA_F ACTOR  <  0.001)) 

{  confirm_with_user  () ; } 
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A  quick  test  can  be  conducted  to  verify  implementation  of  the  hyper-time  algorithm.  In  real-time 
simulation  (hyper-time  factor  equals  one),  a  pilot  flies  from  a  known  initial  position  at  a  constant 
heading  and  airspeed  and  notes  the  aircraft’s  location  after  one  minute  of  clock  time.  This  flight 
is  repeated  after  implementing  a  hyper-time  factor  of  2.0.  The  pilot  should  reach  the  same 
location  after  30  s  of  clock  time. 

2.4.2  Potential  problems  in  implementing  the  hyper-time  algorithm. 

•  Multiple  locations  or  forms  for  delta_t.  Often,  the  value  for  the  time  integration  factor 
is  assigned  as  a  variable  such  as  delta_t  and  all  calculations  which  use  time  refer  to  the  variable 
rather  than  to  a  specific  value.  However,  it  is  also  possible  that  there  is  no  single  location  for  the 
time  integration  and  that  the  source  code  must  be  scrubbed  for  all  instances  of  time.  The  time 
integration  factor  may  also  be  found  in  multiple  forms.  Delta_t  may  be  a  constant  in  a  first-order 
equation  but  also  buried  as  a  coefficient  to  be  used  in  a  difference  equation.  This  is  a  particular 
problem  for  legacy  systems  which  may  contain  several  generations  of  code. 

•  Multiple  processors.  If  a  simulation  system  incorporates  multiple  processors,  and  the 
designer  did  not  utilize  a  master  clock,  the  hyper-time  algorithm  must  be  implemented  for  each. 

It  may  be  possible,  however,  to  implement  above  or  less  than  real-time  training  in  such  a  system 
by  limiting  the  scope  of  operations  for  ARTT.  For  example,  if  flight  operations  are  incorporated 
into  one  processor  and  ground  operations  onto  another,  it  should  be  possible  to  implement  ARTT 
limited  to  flight  operations  only  so  that  ARTT  is  stable  only  while  the  aircraft  is  in  flight.  The 
separate  processes  and  time  constants  used  for  simulating  the  landing  gear  would  produce 
catastrophic  instabilities  if  the  pilot  attempted  to  taxi,  take  off,  or  land.  This  limitation  may  or 
may  not  be  a  problem  depending  on  training  objectives. 

Networked  simulators.  ARTT  within  a  network  of  simulators  operating  under 
Distributed  Interactive  Simulation  (DIS)  protocols,  has  been  accomplished  by  implementing  the 
hyper-time  algorithm  on  each  simulator  (Guckenberger  et  al,  1 995).  This  demonstration  was 
conducted  at  the  1994  Industry/Interservice  Training  Systems  and  Education  Conference 
(I/ITSEC).  Four  flight  simulators,  a  computer-generated  fighter  aircraft,  and  a  DIS  stealth 
platform  were  modified  using  the  hyper-time  algorithm  to  operate  at  three  times  real  time.  The 
different  simulators  used  different  host  computers  and  operating  systems  and  required  unique 
adaptations  of  the  hyper-time  algorithm.  However,  in  all  cases,  the  modifications  required  less 
than  one  hour.  Additional  modifications  were  required  to  insure  that  the  DIS  dead  reckoning 
algorithms  correctly  incorporated  changes  to  simulated  time.  No  difficulties  with  cross-platform 
compatibility,  network  stability,  or  interactions  among  players  such  as  missile  fly-outs  were 
observed.  More  extensive  research  is  required  to  determine  whether  above  real-time 
modifications  could  be  successfully  implemented  on  a  more  complex  network.  Also,  procedures 
must  be  established  to  assign  a  hyper-time  factor  as  a  component  of  exercise  setup  or  debrief. 
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3.  EXPERIMENT  1 :  ARTT  FOR  AIR  COMBAT  SKILLS 
TRAINING  WITH  EXPERIENCED  PILOTS 


Previous  research  suggests  that  ARTT  should  be  an  effective  training  strategy  for:  (a)  tasks 
which  require  significant  effort  at  time  and  workload  management  and,  (b)  trainees  who  have 
completed  the  cognitive  portions  of  skill  acquisition.  In  Experiment  1,  the  tasks  selected 
required  the  pilot  to  employ  the  F-16’s  air-to-air  radar  for  beyond-visual-range  air  combat  and  to 
conduct  air  intercepts.  The  trainees  were  active-duty  F-16  pilots  who  are  trained  in  air  combat 
but  had  few  recent  opportunities  to  practice  using  these  skills.  The  questions  evaluated  in  this 
experiment  were: 

1 .  Is  above  real-time  training  as  effective  as  real-time  training? 

2.  Will  above  real-time  training  produce  better  transition  to  more  difficult  tasks  than  normal 
time  training? 

3 .  How  will  fewer  clock  hours  of  above  real-time  training  compare  to  conventional,  real¬ 
time  training? 


3.1  Research  methods 

3.1.1  Overview.  Volunteer  F-16  pilots  participated  in  a  simulated  training  session  with  two 
tasks:  radar  skills  and  air  intercepts.  In  the  first  task,  pilots  practiced  using  air-to-air  radar  to 
search  and  sort  radar  contacts  and  to  build  a  picture  of  bandit  aircraft  positions  and  actions.  Pilots 
were  instructed  to  fly  their  F-16  straight  and  level,  directly  into  a  bandit  formation.  Pilots  were 
to  use  their  radar  to  identify  the  altitude,  airspeed,  position,  and  actions  of  two  to  six  aircraft. 

The  second  task  required  the  pilot  to  fly  intercepts  using  the  air-to-air  radar.  The  pilot  was 
instructed  to  search  and  sort  die  bandit  formations,  select  the  trailing  threat  as  his  target,  offset 
appropriately,  and  perform  a  stem  conversion.  An  intercept  was  scored  as  successful  if  the  pilot 
was  able  to  roll  out  inside  a  60°  cone  of  the  bandit's  tail,  within  two  nautical  miles,  and  with  a 
positive  rate  of  closure.  For  both  the  radar  skills  and  intercept  tasks,  training  scenarios  were 
presented  in  building  block  format  with  two  simple  scenarios  followed  by  two  scenarios  of 
medium  difficulty  and  finally  three  difficult  scenarios.  Task  difficulty  was  increased  by 
increasing  the  number  of  bandits  from  two  to  six  and  increasing  the  complexity  of  bandit  actions. 
The  experimental  sessions  were  conducted  by  a  former  F-16  instructor  pilot  and  squadron 
commander  who  acted  in  the  role  of  evaluator  during  training.  After  the  participating  pilot 
completed  each  scenario,  the  pilot  debriefed  the  evaluator  regarding  scenario  content,  actions, 
positions,  and  final  parameters.  The  pilot  was  then  informed  if  any  responses  were  incorrect. 

The  evaluator  scored  the  pilot's  performance  during  run  time  and  a  second  time  during 
debriefing.  After  each  pilot  completed  the  training  trials,  he  flew  five  test  scenarios  for  both  the 
radar  skills  and  intercept  tasks.  The  test  scenarios  were  designed  to  be  more  difficult  in  terms  of 
complexity  and  workload  than  any  of  the  training  scenarios.  The  evaluator  scored  each  test 
scenario  during  run-time  and  during  the  pilot’s  debrief.  The  evaluator  did  not  provide  any 
feedback  to  the  pilot  during  test  trials.  Half  of  the  subjects  trained  at  real  time  and  half  trained  at 
1.5  times  real  time.  All  pilots  were  tested  at  real  time.  Pilots  were  informed  that  the  experiment 
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was  an  evaluation  of  ARTT  but  they  were  not  informed  whether  they  were  to  receive  real-time 
training  (RTT)  or  ARTT. 


3.1.2  Participants.  The  participants  in  this  effort  were  1 4  active  duty.  Air  Force  F- 1 6C 
pilots  from  the  347th  Fighter  Wing  at  Moody  AFB,  GA.  F-16  experience  ranged  from  150  h  to 

1 600  h  with  a  mean  of  939.  Pilots  who  volunteered  for  this  experiment  received  temporary  duty 
expenses  from  Armstrong  Laboratory. 

3.1.3  Apparatus.  An  F-16  trainer  developed  by  the  ECC  International  Corporation, 

Orlando  FL,  was  selected  for  this  experiment.  The  ECC  F-16  simulator  was  developed  for  the 
Air  Force  Unit  Training  Device  program  and  later  modified  for  ARTT  research.  The  ECC 
simulator  incorporates  F-16  aerodynamics  and  avionics  capabilities  with  a  three-screen,  out-the- 
window,  visual  display  system.  The  system  has  the  capability  to  present  scenarios  in  which  other 
aircraft  fly  in  pre-recorded  flight  paths. 

3.1.4  Procedure.  Pilots  were  randomly  assigned  to  the  RTT  or  ARTT  group.  On  arrival  at 
ECC,  pilots  were  briefed  on  the  nature  of  the  evaluation  and  completed  a  consent  form.  All 
pilots  performed  the  radar  skills  task  first  followed  by  the  intercept  task 

3. 1 .4. 1  Radar  Skills.  The  objective  of  radar  skills  training  was  to  increase  pilot 
proficiency  in  using  air  radar  to  search  and  sort  multiple,  maneuvering  targets.  For  the  radar 
skills  task,  pilots  received  the  following  instructions: 

During  training  and  testing  of  the  radar  skills  task,  fly  your  F-16  on  a  straight  and 
level  course.  All  bandits  in  the  scenario  will  fly  preplanned  routes.  Your  task  is  to 
use  the  radar  as  effectively  and  efficiently  as  possible  to  provide  you  with  all  of  the 
critical  information  concerning  the  inbound  bandits.  When  performing  the  radar 
skills  tasks,  your  performance  and  the  evaluator's  grading  will  depend  on  your  ability 
to: 


Search  all  airspace  (surface  to  50  k)  before  the  closest  bandit  is  within  40nm 
Know  the  initial  picture  (number  of  bandits,  formation,  aspect,  altitude,  airspeed). 
Determine  bandit  actions. 

Monitor  actions. 

Know  the  picture  at  20  nm. 

Know  which  bandit  is  the  highest  threat. 

Pilots  were  further  briefed  that  the  bandits  were  MiG-23  s  in  air-to-ground  roles  as  bombers. 
The  pilot's  F-16  was  initialized  at  15,000  ft,  450  knots  airspeed,  and  heading  360°.  Other  initial 
conditions  are  specified  in  Appendix  C. 
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The  radar  skills  portion  of  the  evaluation  began  with  a  relatively  simple  scenario  (see  Fig.  1). 
The  pilot  was  to  call  out  radar  contacts  and  bandit  actions  as  they  occurred.  The  evaluator  stood 
just  behind  the  cockpit  and  could  view  the  radar  screen  on  a  video  monitor.  The  run-time  scores 
were  recorded  on  the  data  sheet  in  Appendix  D. 


Figure  1.  Simple  scenario  from  the  radar 
skills  task. 
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Figure  2.  Moderately  complex  scenario 
from  the  radar  skills  task. 
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Figure  3.  Difficult  scenario  from  the  radar 
skills  task. 


Figure  4.  Test  scenario  (most  difficult) 
from  the  radar  skills  task. 


The  pilot's  radar  skills  performance  was  scored  on  a  scale  of  0  -  3  for  each  of  four  sub-tasks: 
search  the  airspace  and  sample  the  contacts;  sort  the  formations  and  monitor  actions;  describe 
picture  at  20  nm;  and,  target  the  highest  priority  threat  before  coming  within  10  nm.  Grading 
criteria  were: 


3:  Performance  is  correct 
2:  Performance  is  essentially  correct  but  late 
1 :  Performance  is  only  partially  correct 
0:  Totally  incorrect. 

The  pilot’s  scores,  0  -  3,  for  each  of  the  four  sub-tasks  were  summed  for  a  run-time  score  of 
0-12. 


13 


After  passing  the  bandit  formation,  the  simulation  stopped  and  the  evaluator  asked  the  pilot 
to  debrief  the  scenario.  Debrief  consisted  of  four  sub-tasks:  describe  the  initial  picture,  bandit 
actions,  the  picture  at  20  nm,  and  the  factors  used  to  determine  the  highest  threat.  Debrief 
performance  was  scored  using  the  same  scale  as  for  the  run-time  scores.  The  pilot’s  scores,  0-3, 
for  each  of  the  four  sub-tasks  were  summed  for  a  debrief  score  of  0  -  1 2.  After  the  pilot  had 
completed  his  debrief,  the  evaluator  provided  feedback  on  the  scenario. 

After  completing  two  relatively  simple  scenarios,  the  pilot  flew  two  moderately  complex 
scenarios  followed  by  three  more  complex  scenarios  (see  Fig.  2  and  3).  At  the  completion  of 
training,  the  pilot  completed  a  NASA  TLX  questionnaire  which  measures  subjective  workload 
(Appendix  E).  After  completing  the  TLX,  the  pilot  was  informed  that  the  next  five  scenarios 
would  constitute  the  test  phase.  These  scenarios  were  designed  to  be  more  difficult  than  any  of 
the  training  scenarios  (see  Fig.  4).  No  feedback  was  offered  after  the  test  scenarios.  All  pilots 
were  tested  in  real  time,  however,  the  ARTT  pilots  were  not  informed  that  simulated  time  had 
changed.  Test  scenarios  were  scored  as  during  training.  After  completing  radar  skills  test,  the 
pilot  completed  a  second  TLX  and  then  took  a  break. 

3. 1.4.2  Intercept  Task.  The  intercept  task  was  similar  to  the  radar  skills  task  except  that 
the  pilot  was  instructed  to  target  the  trailing  threat,  select  an  offset,  and  perform  a  stem 
conversion  in  order  to  visually  identify  the  bandit.  Run-time  scores  were  recorded  for  seven  sub¬ 
tasks:  search  airspace,  sample  contacts,  sort  formation  and  monitor  action,  target  the  trailing 
threat,  obtain  proper  offset,  convert  to  stem  (within  30°  and  6000  ft)  and,  shoot  within 
parameters  (see  Appendix  D).  The  evaluator  used  the  same  grading  criteria  as  for  the  radar  skills 
task.  The  pilot’s  scores  (0  -  3)  for  each  sub-task  were  summed  for  a  run-time  score  of  0  -  21 . 
After  completing  each  intercept,  the  pilot  debriefed  the  evaluator.  Debrief  consisted  of  four  sub¬ 
tasks:  describe  the  initial  picture,  bandit  actions,  the  picture  at  20  nm,  and  the  factors  used  to 
determine  the  highest  threat.  Debrief  performance  was  scored  using  the  same  criteria  as  for  the 
run-time  scores.  The  pilot’s  scores,  0  -  3,  for  each  of  the  four  sub-tasks  were  summed  for  a 
debrief  score  of  0  - 12.  After  the  pilot  had  completed  his  debrief,  the  evaluator  provided  feedback 
on  the  scenario.  Pilots  flew  intercept  scenarios  which  were  arranged  in  building  block  format  as 
with  the  radar  skills  task;  i.  e.,  two  simple  trials,  two  moderately  complex  trials,  followed  by 
three  difficult  trials.  After  completing  intercept  training,  pilots  completed  a  third  TLX  workload 
questionnaire  and  then  began  the  test  phase.  The  test  phase  consisted  of  five  intercepts  presented 
at  real  time  for  all  pilots  without  feedback  from  the  evaluator.  Immediately  following-the  last 
intercept  test  scenario,  a  final  TLX  questionnaire  was  administered. 

Following  completion  of  the  intercept  phase,  the  experiment  concluded  with  a  final  debriefing  to 
the  pilot  explaining  the  specifics  of  the  evaluation.  Total  time  for  this  experiment  was 
approximately  three  hours  providing  all  systems  remained  operational. 

3.2  Results 

3.2.1  Radar  Skills  Task  Performance.  Run-time  and  debrief  scores  for  the  radar  skills  task 
were  transformed  to  percent  of  maximum  possible  score.  Scores  for  training  and  test  trials  were 
pooled  into  three  blocks  of  scenario  difficulty:  training  trials  1-4  which  were  scenarios  of  simple 
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to  moderate  complexity,  training  trials  5-7  which  were  difficult  scenarios,  and  test  trials.  Mean 
percent  run-time  scores  and  debrief  scores  for  these  three  blocks  of  trials  (levels  of  scenario 
difficulty)  are  plotted  on  Figures  5  and  6. 


Scenario  diffiaity 

Figure  5.  Mean  run-time  percent  scores  for  radar  skills  task. 


Scenario  difficulty 

Figure  6.  Mean  debrief  percent  scores  for  radar  skills  task. 

Test  performance  between  the  RTT  and  ARTT  groups  was  not  significantly  different  for 
either  the  run-time  scores,  xRrr  =  79%,  Xartt  =  78%,  ns,  or  for  debrief  scores,  xRTr  =  80%, 
xMn  =  82%,  ns.  There  is  a  significant  interaction  between  training  group  and  scenario 
complexity  for  both  run-time  scores,  F(2,24)  =  6.39,  p  =  .006,  and  for  debrief  scores,  F(2,24)  = 
11.24,  p<  .001. 

For  run-time  scores  (Fig.  5),  radar  skills  performance  for  pilots  trained  in  real  time  did  not 
change  significantly  as  scenario  complexity  increased,  F  <  1.  However,  run-time  scores  for 
pilots  trained  using  ARTT  changed  significantly  across  trials,  F(2,24)  =  11.29,  p<  .001.  Least 
significant  difference  (LSD)  tests  show  that  run-time  scores  for  difficult  trials  were  significantly 
lower  than  scores  for  simple  and  moderate  trials,  t(24)  —  3. 75,  p  =  .001,  and  for  test  trials,  t(24) 
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-  3. 70,  p  <  .001.  Scores  for  simple  and  moderately  complex  trials  were  not  significantly 
different  from  scores  for  test  trials,  t(24)  -0.07. 

Debrief  scores  (Fig.  6)  show  significant  change  across  levels  of  scenario  difficulty  for  both 
pilots  trained  in  real  time,  F(2,24)  =  3.60,  p  =  .043,  and  ARTT,  F(2,24)  =  31.08,  p<.001.  For 
pilots  trained  in  real  time,  LSD  tests  show  that  there  is  a  significant  decrease  in  debrief  scores 
between  the  difficult  training  trials  and  the  test  trials,  t(24)  =  2.87,  p  =  .008.  The  differences  in 
debrief  scores  for  pilots  trained  in  real  time  between  the  simple-moderately  complex  trials  and 
the  difficult  trials,  t(24)  =  -1.12,  p  =  .28,  or  between  simple-moderate  and  test  trials  t(24)  =  1.0, 
p  =  .33)  are  not  significant.  For  pilots  trained  using  ARTT,  there  is  a  significant  decrease  in 
debrief  scores  between  simple-moderately  complex  trials  and  difficult  trials,  t(24)  =  3.32,  p  = 
.003,  and  a  significant  increase  in  scores  between  the  difficult  training  trials  and  test  trials,  t(24) 
=  -3. 78,  p  =  .001.  The  difference  between  debrief  scores  on  the  simple-moderate  training  trials 
and  test  trials,  t(24)  =  0.45  is  not  significant. 

3.2.2  Intercept  Task  Performance.  One  pilot  was  unable  to  complete  the  intercept  task  due 
to  equipment  failure.  Run-time  and  debrief  scores  for  the  intercept  task  were  transformed  to 
percent  of  maximum  possible  score.  Scores  for  training  and  test  trials  were  pooled  into  three 
blocks  of  scenario  difficulty:  training  trials  1-4 which  were  scenarios  of  simple  to  moderate 
complexity,  training  trials  5-7  which  were  difficult  scenarios,  and  test  trials.  Mean  percent  run¬ 
time  scores  and  debrief  scores  for  these  three  blocks  of  trials  (levels  of  scenario  difficulty)  are 
plotted  on  Figures  7  and  8. 

Test  performance  between  the  RTT  and  ARTT  groups  was  not  significantly  different  for 
either  run-time  scores,  xRJT  =  74%,  Xartt  =  73%,  ns,  or  debrief  scores,  xRn  =  71%,  xAR7T  = 
85%,  ns.  There  were  no  significant  differences  among  the  intercept  run-time  scores  (Fig.  7)  due 
to  training  group,  F  <  1,  level  of  scenario  difficulty,  F  <  1,  or  the  group  by  difficulty  interaction, 
F  (2,  22)  =1.21,  ns.  There  were  no  significant  differences  among  the  intercept 


Scenario  diffiaJty 

Figure  7.  Mean  run-time  percent  scores  for  intercept  task. 


16 


Scenario  diffiaity 

Figure  8.  Mean  debrief  percent  scores  for  intercept  task. 


debrief  scores  (Fig.  8)  due  to  training  group,  F  <  1 ,  or  the  training  group  by  level  of  difficulty 
interaction,  F  - 1.  There  were  significant  differences  in  debrief  scores  among  the  levels  of 
scenario  difficulty,  F(2,22)  =  10.45,  p  =  .001.  LSD  tests  show  that  debrief  scores  significantly 
decreased  between  the  simple-moderately  complex  trials  and  the  difficult  trials,  t(22)  =  4.46,  p  < 
.001,  and  significantly  increased  between  the  difficult  training  trials  and  test  trials,  t(22)  =  -2.86, 
p  <  .001.  The  difference  between  debrief  scores  for  the  simple-moderate  trials  and  test  trials 
was  not  significant,  t(22)  =  1.59,  ns. 

3.2.3  TLX  ratings.  Pilots  completed  a  TLX  workload  rating  form  (Appendix  E)  after 
radar  skills  training,  radar  skills  testing,  intercept  training,  and  intercept  testing.  This  form  asked 
pilots  to  rate  the  perceived  effect  of  six  different  demands  (workload  categories)  on  their 
performance:  physical  demands,  time  pressure,  interpreting  radar,  switchology,  flying  the 
aircraft,  and  frustration.  TLX  ratings  were  compared  by  training  group  (RTT  vs.  ARTT),  TLX 
workload  category,  and  time  of  rating.  Training  group  did  not  significantly  affect  overall 
workload  ratings,  F(l,  11)  =  1.21,  ns.  Workload  ratings  were  also  not  significantly  affected  by 
the  interactions  of  training  group  with  workload  category,  F(5,  55)  =  1.05,  ns;  training  group 
with  time  of  rating,  F  <  1;  or  training  group  by  workload  category  by  time  of  rating,  F(15,  1 65) 

=  1. 71,  ns.  Workload  ratings  were  significantly  influenced  by  the  interaction  of  workload 
category  and  time  of  rating,  F(15, 165)  =  3.93,  p  <  .001.  Mean  TLX  ratings  for  the  different 
workload  categories  and  rating  times  are  presented  on  Figure  9.  Of  the  six  workload  categories, 
only  flying  the  aircraft,  F(3,  36)  =  9.96,  p  <  .001,  and  frustration  level,  F(3,36)  =  5.89,  p  =  .002, 
were  significantly  affected  by  time  of  rating.  For  both  of  these  demands,  there  was  a  significant 
increase  in  rated  workload  between  the  radar  skills  task  and  the  intercept  task:  flying  the  aircraft, 
F(l,12)  =  11.99,  p  =  .005,  frustration  level,  F(l,12)  -  8.39,  p  =  .013. 
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Figure  9.  Mean  TLX  ratings  for  each  workload  category  and  time  of  testing. 


3.3  Discussion 


The  questions  which  were  evaluated  in  this  research  were: 

1 .  Is  above  real-time  training  as  effective  as  real-time  training? 

2.  Will  above  real-time  training  produce  better  transition  to  more  difficult  tasks  than  normal 

time  training? 

3.  How  will  fewer  clock  hours  of  above  real-time  training  compare  to  conventional,  real¬ 
time  training? 

3.3.1  Training  effectiveness  of  ARTT.  RTT  and  ARTT  resulted  in  equal  performance  on 
the  real-time  test  trials  for  both  the  radar  skills  and  intercept  tasks,  however,  performance  on 
training  trials  for  the  radar  skills  task  was  degraded  by  ARTT.  For  the  simple  and  moderate 
scenarios,  the  ARTT  pilots  were  able  to  maintain  their  performance  while  coping  with  the 
increased  time  pressure.  When  the  task  became  more  difficult,  their  performance  scores 
decreased.  Viewed  from  this  perspective,  ARTT  may  save  training  time  but  at  the  cost  of  poorer 
performance  in  the  training  simulator  for  the  most  demanding  tasks. 

3.3.2  Transition  to  more  demanding,  real-time  tasks.  The  rationale  for  ARTT,  however,  is 
more  than  reducing  training  time.  The  experience  of  NASA  test  pilots  who  first  used  ARTT  was 
that  the  demands  of  ARTT  in  the  simulator  provided  better  training  for  flying  the  actual  aircraft 
than  normal  simulator  training.  One  hypothesis  is  that  simulator  training  is  inherently  less 
demanding  than  actual  flight  and  that  ARTT  replaces  some  of  the  demands  of  flight  with 
additional  time  pressure.  This  evaluation  mimics  the  transition  from  simulator  to  aircraft  by  a 
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within-simulator  transfer  of  training  design  in  which  task  demands  increase  from  training  to  test 
conditions  (see  Fig.  1  -  4).  Test  scenarios  were  more  complex  than  any  of  the  training  scenarios 
and  testing  was  conducted  in  normal  time.  Pilots  who  trained  in  real  time  showed  no  significant 
change  in  performance  from  training  to  test  trials  for  radar  skills  run-time  scores  (Fig.  5)  and 
showed  a  significant  decrease  in  performance  for  debrief  scores  (Fig.  6).  ARTT  pilots  improved 
performance  on  the  radar  skills  task  when  transitioning  from  above  real  time  to  normal  time  even 
though  the  test  scenarios  were  more  complex  than  the  scenarios  used  in  training  (Fig.  5  and  6). 
However,  test  trial  performance  of  pilots  trained  using  ARTT  was  not  significantly  better  than 
performance  of  pilots  trained  in  real-time.  While  ARTT  trained  pilots  performed  the  test  trials  as 
well  as  the  real-time-trained  pilots,  there  is  no  support  for  the  hypothesis  that  ARTT  provided 
better  training  than  conventional,  real-time  simulator  training. 

This  conclusion  is  further  supported  by  the  TLX  workload  rating  data.  Based  on  the  NASA 
experience  with  ARTT,  it  would  be  predicted  that  ARTT  pilots  would  report  lower  levels  of 
workload,  particularly  time  pressure,  when  transitioned  from  above  real-time  training  trials  to 
real-time  test  trials.  The  interaction  effects  of  training  group  with  time  of  rating  or  of  training 
group  by  time  of  rating  by  workload  category  were  not  significant.  The  only  factor  which 
significantly  affected  workload  ratings  was  the  transition  from  the  radar  skills  task  in  which 
pilots  were  instructed  to  fly  straight  and  level  to  the  intercept  task  in  which  pilots  performed  an 
offset  and  stem  conversion.  For  the  intercept  task,  all  pilots  reported  significant  increases  in 
effort  for  flying  the  aircraft  and  in  frustration  level.  Using  the  present  experimental  protocol  in 
which  pilots  immediately  transitioned  from  ARTT  to  real-time  test,  there  were  no  reported 
decreases  in  workload. 

The  results  from  the  intercept  task  show  that  ARTT  may  not  be  beneficial  for  all  training 
problems.  Unlike  the  radar-skills  task,  there  is  no  training-group  by  scenario  difficulty 
interaction  for  the  intercept  task  (Fig.  7  and  8)  even  though  the  intercept  scenarios  progressed 
from  simple  to  difficult  in  the  same  manner  as  the  radar  skills  scenarios.  Further,  debrief  scores 
for  intercept  training  show  a  significant  decrease  for  both  RTT  and  ARTT  pilots  as  the  scenarios 
become  more  demanding.  Test  performance  was  not  significantly  different  for  either  run-time  or 
debrief  scores  between  the  RTT  and  ARTT  groups  nor  was  there  a  significant  change  in 
performance  scores  from  training  to  test.  For  the  intercept  task,  pilot  performance  seems  to  be 
insensitive  to  the  demands  of  above  real-time  training.  For  all  pilots,  performance  was  highly 
sensitive  to  increasing  task  complexity.  Review  of  the  exact  procedures  used  in  the  experiment 
is  required  to  understand  the  discrepancy  between  the  results  of  the  two  tasks.  All  pilots 
participated  in  the  radar-skills  task  before  the  intercept  task.  Therefore,  pilots  had  over  an  hour's 
practice  at  using  their  air-to-air  radar  before  beginning  to  fly  intercepts.  All  pilots  were  therefore 
able  to  use  their  radar  effectively  to  build  the  picture  of  bandit  actions.  The  evaluator  who 
observed  and  scored  pilot  performance  in  this  research  noted  that  the  variability  in  intercept 
scores  did  not  result  from  differences  in  time  management  or  radar  employment  but  from 
differences  in  selecting  an  offset.  Pilots  who  selected  an  offset  which  provided  a  tactical 
advantage  nearly  always  received  high  scores  while  pilots  who  selected  a  less  advantageous 
offset  or  who  offset  at  the  wrong  time  received  poorer  scores. 


19 


The  radar  skills  data  show  that  ARTT  can  be  as  effective  as  real-time  training  for  tasks  which 
emphasize  time  and  workload  management.  The  intercept  data  show  that  ARTT  is  probably 
much  less  effective  for  tasks  which  emphasize  other  skills,  in  this  case,  tactical  decision  making. 
While  ARTT  may  be  effective  for  training  a  pilot  to  quickly  gather  and  integrate  information, 
other  instructional  strategies  such  as  case  study,  video  debrief,  or  freeze  and  reset  may  be  more 
effective  for  teaching  decision  making  skills. 

3.3.3  Effect  of  ARTT  on  time  required  for  training.  The  final  issue  is  whether  ARTT  saves 
training  time,  i.e.,  clock  hours  in  the  simulator.  Given  that  the  simulator  was  operating  at  normal 
time  or  1.5  times  normal  time,  the  ARTT  group  received  only  two  thirds  the  time  in  the 
simulator  of  the  RT  group.  Since  test  trial  performance  was  not  significantly  affected  by  training 
group,  the  hypothesis  that  ARTT  can  provide  effective  training  with  fewer  clock  hours  of 
simulator  time  is  supported. 

3.4  Experiment  1:  Conclusions 

1 .  ARTT  was  more  difficult  than  conventional,  real-time  simulator  training.  The  effect  of 
increasing  time  pressure  was  poorer  task  performance  in  the  simulator  particularly  for  demanding 
tasks. 

2.  Pilots  trained  using  ARTT  were  able  to  perform  a  very  demanding  task  as  well  as  pilots 
trained  in  normal  time  even  though  they  received  only  two  thirds  of  the  simulator  time. 

3.  The  effect  of  ARTT  was  dependent  on  the  skills  required  to  perform  a  task.  In  this 
experiment,  ARTT  had  greater  effect  on  a  task  which  required  information  gathering, 
switchology,  radar  interpretation,  and  time  and  workload  management.  A  task  that  required 
tactical  decision  making  was  unaffected  by  ARTT. 
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4.  EXPERIMENT  2:  ARTT  FOR  EMERGENCY  PROCEDURES 
TRAINING  WITH  EXPERIENCED  PILOTS 


Experiment  1  demonstrated  that  ARTT  at  1.5  times  real  time  supported  training  some  of  the 
skills  necessary  for  effective  air  combat  as  well  as  real-time  training  with  fewer  clock-hours  of 
simulator  time.  A  second  interest  of  the  Human  Systems  Center’s  TPIPT  is  the  potential 
application  of  ARTT  to  emergency  procedures  (EP)  training.  Simulator-based  training  for 
emergency  procedures  allows  the  pilot  to  practice  performing  tasks  which  may  never  be 
performed  in  the  aircraft  but  if  required,  must  be  performed  correctly  and  promptly.  ARTT  may 
enhance  simulator  training  for  emergencies  by  increasing  automaticity  and  decreasing  workload. 
The  result  should  be  faster  responses  to  emergencies  and  better  performance  on  a  secondary  task. 

4.1  Research  methods 

4.1.1  Overview.  For  this  experiment,  pilots  conducted  single-ship,  defensive  counter-air 
missions  over  a  ground  target  using  two  scenarios.  In  one  scenario,  single  emergency,  the  pilot’s 
aircraft  suffered  engine  failure.  The  pilot’s  task  was  to  restart  the  engine  and  then  to  engage  an 
incoming  bandit.  In  the  other  scenario,  multiple  emergencies,  the  pilot  has  to  clear  an  indication 
that  equipment  was  overheating,  restart  a  failed  engine,  respond  to  a  warning  light,  and  engage 
two  bandits  in  succession.  Pilots  received  initial  training  in  real  time  followed  by  additional 
practice  in  real  time  or  at  1.5  times  real  time.  All  pilots  were  then  tested  in  real  time.  The 
dependent  measures  were  time  required  to  correct  emergencies  and  time  required  to  kill  the 
bandits. 

4.1.2  Participants.  The  participants  in  Experiment  2  were  12  of  the  14  pilots  who 
participated  in  Experiment  1 .  Two  pilots  were  unable  to  complete  Experiment  2  due  to 
equipment  failure. 

4.1.3  Apparatus.  The  F- 1 6  simulator  used  in  Experiment  1  was  also  used  in  Experiment  2. 
The  bandit  aircraft  were  flown  by  an  automated  threat  system  integral  to  the  ECC  F-16  trainer. 

4.1.4.  Procedure.  Pilots  were  randomly  assigned  to  either  the  RTT  or  ARTT  condition.  All 
pilots  received  familiarization  training  in  the  ECC  F-16  simulator  in  real  time.  Familiarization 
consisted  of  flying  vertical-S  maneuvers,  90°  turns,  and  loops.  After  familiarization,  pilots  were 
trained  and  tested  in  one  of  the  two  emergency  conditions  (single  or  multiple)  selected  at 
random.  After  a  break,  the  pilot  was  trained  and  tested  in  the  other  condition.  For  both 
conditions,  pilots  received  initial  training  in  real  time  until  they  could  complete  the  task  without 
failures  or  re-starts.  Typically,  initial  training  required  three  or  four  trials.  Data  from  these  trials 
were  not  analyzed.  Six  additional  practice  trials  were  then  conducted  in  real  time  or  above  real 
time.  Finally,  all  pilots  received  four  real-time,  test  trials  using  the  same  scenarios  as  used  in 
training. 

4. 1.4.1  Single  emergency.  Pilots  were  initialized  at  2000ft  above  ground  level  (AGL) 
and  480  knots  inbound  toward  a  power  plant  which  they  were  tasked  to  defend  against  air 
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assault.  Shortly  after  the  trial  started,  the  F-16’s  engine  failed  and  the  pilot  had  to  restart  using 
established  procedures.  Immediately  after  restart,  the  pilot  engaged  a  MiG-29.  The  trial  ended 
when  the  F- 16  pilot  or  the  bandit  was  killed.  Dependent  variables  were  time  to  restart  engine  and 
time  to  kill  the  bandit. 

4. 1.4.2  Multiple  emergency.  The  multiple  emergency  task  was  similar  to  the  single 
emergency  except  the  pilot  first  had  to  check  an  equipment  hot  indicator  light,  restart  a  failed 
engine,  acknowledge  another  indicator  light  (hydraulic  failure),  and  then  engage  a  MiG-29 
followed  by  a  second  MiG-29.  Dependent  variables  were  time  to  respond  to  the  equipment  hot 
light,  time  to  restart  the  engine,  time  to  respond  to  the  hydraulic  failure  indicator  light,  and  time 
to  kill  both  bandits. 

4.2  Results 


4.2.1  Single  emergency.  Mean  time  to  restart  the  engine  and  time  required  to  kill  the 
bandit  on  test  trials  are  plotted  on  Figure  10. 
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Figure  10.  Mean  time  to  restart  engine  kill  bandit  for  test  trials. 


Test  performance  on  both  variables  was  not  affected  by  training  condition.  Time  required  to 
restart  the  failed  engine,  t(ll)  =  0.08,  ns,  and  time  required  to  kill  the  bandit,  t(ll)  =  0.4,  ns, 
were  not  significantly  different  between  the  RTT  and  ARTT  pilots. 


4.2.2  Multiple  emergency.  Mean  time  on  test  trials  to  respond  to  the  equipment  hot  light, 
time  to  restart  the  failed  engine,  time  to  respond  to  the  hydraulic  failure  light,  and  time  to  kill 
both  bandits  are  plotted  on  Figure  1 1 .  For  performance  on  test  trials,  time  to  clear  the  equipment 
hot  light  was  not  significantly  affected  by  training  condition,  t(ll)  =  1.65,  ns.  However,  times 
required  to  restart  the  failed  engine,  t(ll)  =  -4.54,  p  <  .001,  respond  to  hydraulic  failure,  t(ll)  = 
-4.93,  p  <  .001,  and  kill  both  bandits,  t(l  1)  =  -2. 76,  p  =  .018,  were  all  significantly  less  for  pilots 
trained  using  ARTT  than  for  pilots  trained  using  RTT. 
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Figure  1 1 .  Mean  time  to  respond  to  multiple  emergencies  on  test  trials. 


4.3  Discussion 


The  ARTT  pilots  in  Experiment  2  replicated  the  training  procedures  used  by  NASA  in  that 
the  pilots  first  experienced  initial  training  in  real-time  followed  by  additional  trials  at  1 .5  times 
real-time.  The  RTT  pilots  in  experiment  2  received  all  training  trials  in  real-time.  The 
experience  of  NASA  pilots  in  the  F-15  RPV  program  was  that  ARTT  provided  better  preparation 
for  highly  demanding  missions  than  real-time  simulation.  In  Experiment  2,  ARTT  was  more 
time  efficient  than  RTT  for  the  single  emergency  task  but  provided  no  other  training  benefit. 
Pilots  in  the  single  emergency  task  performed  engine  restarts  and  defeated  a  bandit  aircraft  as 
quickly  as  pilots  trained  in  real-time  but  no  faster.  However,  in  the  more  complex  multiple 
emergency  task,  pilots  who  practiced  using  ARTT  were  able  to  perform  two  of  the  three  required 
emergency  procedures  and,  killed  both  bandit  aircraft  significantly  faster  than  pilots  who  trained 
in  real  time.  For  the  experienced  F-16  pilots  in  experiment  2,  the  single  emergency  task  which 
consisted  of  restarting  the  engine  and  engaging  a  single  bandit  was  not  especially  demanding. 
ARTT  provided  no  training  advantage  for  this  task  other  than  reducing  the  amount  of  clock-time 
required  to  complete  a  given  number  of  practice  scenarios.  The  multiple  emergency  task  was 
more  demanding  of  the  pilot’s  time  and  workload  management  skills.  For  this  task,  practice 
using  ARTT  after  initial  training  in  real  time  helped  the  pilot  to  perform  the  emergency 
procedures  and  to  successfully  engage  both  bandits  significantly  faster  than  pilots  who  received 
all  of  their  training  in  real  time. 
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4.4  Experiment  2:  Conclusions 

The  results  of  the  Experiment  2  support  the  hypothesis  that  ARTT  provides  improved 
training  for  some  emergency  procedures  compared  to  conventional,  real-time  training.  Pilots 
trained  using  ARTT  performed  responses  in  the  single  emergency  task  as  well  as  pilots  trained  in 
real-time.  ARTT  was  more  efficient  than  real-time  training  in  that  pilots  trained  using  ARTT 
were  able  to  perform  on  test  trials  as  well  as  pilots  trained  in  real  time  but  with  less  training  time. 
For  the  more  demanding  multiple  emergency  task,  pilots  trained  using  ARTT  performed 
emergency  responses  faster  than  pilots  trained  using  RTT  for  three  of  the  four  dependent 
variables.  ARTT  provided  better  training  than  real-time  training  provided  that:  a)  the  tasks  being 
trained  are  highly  demanding  of  a  pilot’s  time  and  workload  management  skills,  and  b)  the  pilot 
has  received  initial  training  in  real-time. 
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5.  EXPERIMENTS:  ARTT  FOR  RADAR  SKILLS 
TRAINING  WITH  STUDENT  PILOTS 


In  Experiments  1  and  2,  the  trainees  were  mission-ready  fighter  pilots  who  were  well  trained 
in  the  tasks  that  were  simulated  but  lacked  recent  experience.  For  these  trainees,  ARTT 
produced  equal  or  better  test  performance  with  less  training  time  than  real-time  simulation.  The 
results  of  Experiments  1  and  2  suggest  that  ARTT  could  increase  time  efficiency  within  a 
squadron  continuation  training  program  designed  to  maintain  or  improve  proficiency  for  seldom 
practiced  skills.  In  Experiment  3,  the  training  benefits  of  ARTT  were  assessed  with  student 
pilots  in  a  Formal  Training  Unit  (FTU).  In  this  experiment,  the  radar-skills  task  from  Experiment 
1  was  used  as  a  supplement  to  an  existing  training  syllabus.  Students  practiced  radar  skills 
within  a  mission  context  after  they  had  successfully  completed  the  air-to-air  portion  of  the  F-16 
FTU  syllabus.  Further,  in  Experiments  1  and  2,  all  pilots  received  the  same  number  of  training 
trials  with  the  ARTT  pilots  receiving  less  time  (clock  hours)  in  the  simulator  than  the  pilots 
trained  in  real  time.  In  Experiment  3,  ARTT  pilots  received  more  training  trials  using 
approximately  the  same  amount  of  clock-time  in  the  simulator  than  the  pilots  trained  in  real  time. 
In  this  respect.  Experiment  3  replicated  the  procedure  used  by  Vidulich,  Yeh,  and  Schneider 
(1983)  who  used  ARTT  to  provide  more  training  in  a  given  time  period  than  could  be  provided 
using  real-time  simulation. 

5.1  Research  methods 

5.1.1  Overview.  The  radar-skills  task  from  Experiment  1  was  modified  for  use  with  student 
pilots.  The  major  changes  were  increasing  the  number  of  training  scenarios  from  7  to  15  and 
eliminating  the  TLX  workload  ratings.  Also,  pilots  assigned  to  the  RTT  condition  received  10 
training  trials  while  pilots  assigned  to  the  ARTT  condition  received  15  training  trials.  Both 
conditions  required  approximately  30  minutes  of  simulator  time.  Experiment  3  was  designed  to 
assess  whether  ARTT  is  an  effective  training  strategy  for  use  with  advanced  student  pilots  and, 
whether  using  ARTT  to  increase  the  number  of  training  trials  will  increase  the  effectiveness  of 
training. 

5.1.2  Participants.  The  participants  in  this  experiment  were  24  students  in  the  F-16C 
training  course  at  Luke  AFB,  A Z.  All  participants  were  new  to  the  F-16  with  between  40  and 
130  F-16  hours.  Of  the  24  pilots,  19  had  no  previous  Air  Force  flying  experience  other  than 
Undergraduate  Pilot  Training  and  Lead-in  Fighter  Training  for  a  total  of  260  to  615  flight  hours. 
The  remaining  pilots  had  previous  assignments  in  other  aircraft  which  were  not  equipped  with 
air-to-air  radar.  These  pilots  had  1500  to  2100  hours  in  other  aircraft  but  only  50  to  100  F-16 
hours.  All  pilots  had  completed  the  air-to-air  portion  of  training  and  had  successfully  completed 
simulator  and  aircraft  sorties  requiring  use  of  the  air-to-air  radar. 

5.1.3  Apparatus.  The  Armstrong  Laboratory  Air  Intercept  Trainer-Plus  (AIT+)  was 
selected  for  Experiment  3.  The  AIT+  is  an  Armstrong  Laboratory  F-16  Air  Intercept  Trainer 
which  has  been  modified  by  replacing  the  computing  hardware  and  software  with  components 
from  the  Armstrong  Laboratory  Multi-Task  Trainer  (Boyle  &  Edwards,  1992).  The  AIT+  is  a 
high-fidelity,  F-16C  simulator  limited  to  air-to-air  operations.  The  AIT+  incorporates  flight, 
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engine,  and  radar  simulations,  with  hands-on-throttle-and-stick  (HOTAS)  controls,  a  radar 
display,  radar  control  panel,  and  a  color  monitor  which  includes  a  heads-up  display  (HUD)  and  a 
limited  out-the-window  display.  For  this  experiment,  the  AIT+  operated  in  autopilot  mode  in 
that  the  aircraft’s  altitude,  airspeed,  and  heading  were  fixed.  The  pilot’s  only  task  was  to  use  the 
radar. 

5.1.4  Procedure.  Pilots  were  randomly  assigned  to  the  RTT  or  the  ARTT  group.  Pilots 
were  given  instructions  on  the  radar  skills  task  as  described  in  section  3. 1.4.1  above  except  that 
the  aircraft  was  flying  on  autopilot.  Also,  the  scoring  procedure  was  modified  slightly.  Rather 
than  asking  the  pilot  to  describe  the  picture  at  20  nm,  pilots  in  Experiment  3  were  asked  to 
describe  the  picture  after  bandit  maneuvers  (actions).  See  Appendix  D  for  sample  score  sheet. 

As  in  Experiment  1,  all  pilots  were  scored  on  two  dependent  measures  for  each  scenario:  scores 
recorded  by  the  evaluator  during  run-time  and,  scores  recorded  by  the  evaluator  while  the  pilot 
debriefed  the  scenario.  Unlike  Experiment  1,  pilots  received  10  or  15  training  scenarios.  Pilots  in 
the  RTT  condition  received  two  relatively  simple  scenarios,  three  moderately  complex  scenarios, 
and  five  complex  scenarios.  Pilots  in  the  ARTT  condition  received  the  same  simple,  moderate, 
and  complex  scenarios  plus  five  additional  complex  scenarios  for  a  total  of  15  training  trials.  All 
pilots  were  tested  in  real  time  on  five  scenarios  which  were  more  complex  than  any  of  the 
training  scenarios.  After  completion  of  the  test  scenarios,  pilots  12  through  24  were  asked  to 
estimate  whether  the  training  trials  were  presented  in  real  time  or  above  real  time  and,  whether 
the  test  trials  were  presented  in  real  or  above  real  time.  Total  time  for  the  experiment  was 
approximately  two  hours. 

5.2  Results 

Scores  for  training  and  test  trials  were  grouped  into  blocks  depending  on  scenario  difficulty. 
Trials  1-5  were  grouped  as  simple-moderate  complexity  scenarios,  trials  6  - 10  as  difficult,  and 
trials  11-15  also  as  difficult  but  for  the  ARTT  group  only.  Trials  16-20  were  test  trials  and 
were  designed  to  be  more  difficult  than  any  of  the  training  trials.  Mean  percent  scores  grouped 
into  blocks  are  plotted  for  run-time  scores  on  Figure  12  and  for  debrief  scores  on  Figure  13.  Test 
performance  was  not  significantly  different  between  the  RTT  and  ARTT  groups  for  run-time 
scores  (xR7 r  =  77.5,  Xartt  =  81. 7,  t(22)  =  1.97,  p  =  .062).  For  debrief  scores,  test  performance 
for  the  ARTT  group  was  significantly  higher  than  for  the  RTT  group  ( xR7T  =  75.8,  Xartt  = 

82.5,  t(22)  =  2.38,  p  =  .026). 

Comparing  scores  on  training  trials  and  test  trials,  run-time  scores  for  the  ARTT  group  show 
significant  increase  from  training  to  test  (F(l,  11)  =  19. 64,  p  =  .001)  while  for  the  RTT  group 
there  is  no  significant  change  in  run-time  scores  from  training  to  test  (F(l,  11)  =  1.44,  p  =  .256) 
see  Figure  12.  For  debrief  scores,  the  ARTT  group  shows  a  significant  increase  from  training  to 
test  (F(l,  11)  =  10.76,  p  =  .007)  while  the  RTT  group  shows  a  significant  decrease  (F(l,  11)  = 

38. 71,  p  <  .000),  see  Figure  13. 
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Figure  12.  Run-time  scores  grouped  by  scenario  difficulty. 
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Figure  1 3.  Debrief  scores  grouped  by  scenario  difficulty. 
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Thirteen  student  pilots  were  asked  to  estimate  whether  training  or  test  trials  were  presented  in 
real-time  or  above  real  time.  Of  the  pilots  trained  in  real-time,  5  out  of  7  correctly  judged  that  the 
training  trials  were  presented  in  real-time;  one  judged  that  training  was  faster  than  real-time  and 
one  could  not  tell.  In  addition,  two  of  these  pilots  believed  that  the  test  trials  were  faster  than 
real-time.  Of  the  pilots  trained  using  ARTT,  2  out  of  6  correctly  judged  that  training  was  faster 
than  real  time,  3  believed  that  training  trials  were  in  real-time  and  one  could  not  tell.  Two  of  the 
pilots  trained  using  ARTT  incorrectly  judged  that  the  test  trials  were  presented  faster  than  real¬ 
time. 


5.3  Discussion 

The  questions  which  were  evaluated  in  this  experiment  were: 

1 .  Will  ARTT  provide  effective  training  for  advanced  student  pilots? 
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2.  Will  additional  training  trials  using  ARTT  provide  a  better  transition  to  a  more  complex  task 

than  the  same  amount  of  time  spent  in  the  simulator  using  real-time  training? 

5.3.1  ARTT  for  student  pilots.  ARTT  will  interfere  with  effective  training  if  the  pace  of 
events  is  so  rapid  that  students  cannot  keep  up.  In  more  formal  terms,  the  training  would 
overload  temporary  (working)  memory  thus  violating  one  of  Schneider’s  (1985)  rules  for  training 
complex  skills.  Evidence  for  this  effect  would  be  that  students  using  ARTT  would  produce 
lower  performance  scores  than  students  using  RTT  and  that  the  ARTT  student  performance 
scores  would  not  improve.  In  this  experiment,  performance  scores  of  students  using  ARTT  were 
indeed  initially  lower  than  scores  of  students  using  RTT  (run  time  percent  scores  for  trials  1-5: 
Xrtt=  79.2,  x  artt  =  73.83,  t(22)  —  2.32.  p  =  .03,  Figure  12;  debrief  percent  scores  for  trials  1-5 
x rtt  =  83.4,  x artt  =  74. 7,  t(22)  =  3.53,  p  =  .002,  Figure  13).  However,  ARTT  student 
performance  significantly  improved  from  trials  1-5  (simple-moderate  scenarios)  to  trials  6-10 
(difficult  scenarios)  for  run-time  (F(l,  11)  =  12.41,  p  =  .005,  Fig.  12),  but  not  for  debrief  scores 
(F(l,  11)  =  3.81,  p  =  .07,  Fig.  13).  Performance  scores  for  students  trained  in  real  time  did  not 
change  significantly  from  the  simple-moderate  scenarios  to  the  difficult  scenarios  for  either  run¬ 
time  (F  <  1,  Figure  12),  or  for  debrief  scores  (F  <  1,  Figure  13).  At  first,  the  student  pilots 
appear  to  have  had  more  difficulty  with  ARTT  than  the  experienced  pilots.  However,  the 
students  overcame  this  deficit  within  five  trials.  In  addition,  pilots  were  unable  to  reliably 
distinguish  real-time  form  above  real-time  simulation  when  they  were  asked  whether  they 
thought  that  the  simulation  had  been  presented  faster  than  real-time.  Overall,  ARTT  at  1 .5  times 
real-time  was  not  a  problem  for  advanced  student  pilots  although  there  was  an  initial 
performance  deficit. 

5.3.2  Using  ARTT  to  provide  additional  training  trials.  In  Experiment  3,  debrief  scores  on 
the  test  trials  were  significantly  higher  for  the  ARTT  group  than  for  the  RTT  group.  Further,  the 
ARTT  group  showed  a  significant  increase  in  run-time  and  debrief  scores  from  the  difficult 
training  trials  to  the  more  complex  test  trials  which  were  presented  in  real  time.  The  RTT  group 
showed  no  change  in  performance  between  training  and  test  trials  for  run-time  scores  (Fig.  12) 
and  a  significant  decrease  for  debrief  scores  (Fig.  1 3).  Pilots  trained  using  ARTT  showed 
increased  scores  when  transitioned  from  a  difficult  task  presented  faster  than  real  time  to  one  that 
is  even  more  demanding  but  presented  in  real  time. 

In  Experiment  1,  ARTT  provided  equal  training  using  less  time  spent  in  the  simulator.  In 
Experiment  3,  ARTT  pilots  received  more  training  trials  than  the  RTT  pilots  while  time  spent  in 
the  simulator  was  approximately  equal  for  both  groups.  The  RTT  group  required  an  average  of 
26.5  minutes  in  the  simulator  to  complete  1 0  scenarios;  the  ARTT  group  required  an  average  of 

30.2  minutes  to  complete  15  scenarios.  The  slightly  longer  time  for  the  ARTT  group  was 
required  because  the  additional  scenarios  were  more  complex  than  the  first  five  scenarios  and 
more  time  was  required  for  bandit  actions.  Overall,  the  idea  of  using  ARTT  to  provide  additional 
training  trials  within  the  same  amount  of  clock  time  as  real  time  training  as  suggested  by 
Vidulich  et  al  (1983)  was  supported.  However,  due  to  the  specific  training  task  used  in  this 
experiment,  the  total  training  time  for  the  two  groups  was  different.  After  each  scenario,  the 
pilot  would  debrief  the  evaluator  describing  the  number,  altitude,  airspeeds,  and  actions  of  bandit 
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aircraft.  As  scenarios  increased  in  complexity,  see  Figures  1  -  3,  the  amount  of  time  required  for 
debrief  increased.  As  a  result,  pilots  in  the  ARTT  group  spent  longer  in  debrief  than  the  RTT 
pilots  leading  to  more  total  training  time  for  ARTT  pilots  than  for  RTT  pilots  (Xrtt=  47  minutes, 
x^rr  =  60  minutes).  Total  training  time  did  not  correlate  with  test  scores  (r  =  +0.11,  ns). 

5.4  Experiment  3:  Conclusions 

1 .  With  advanced  student  pilots  as  trainees,  ARTT  at  1 .5  times  normal  time  initially  produced 
lower  performance  scores  than  real  time  training  even  for  relatively  simple  scenarios.  This 
problem  was  not  observed  with  more  experienced  pilots.  Otherwise,  ARTT  provided  acceptable 
training  for  student  pilots. 

2.  As  with  the  more  experienced  pilots,  ARTT  was  more  difficult  than  real-time  training. 
Performance  scores  during  training  were  lower  for  students  trained  using  ARTT  than  for  students 
trained  in  real  time.  However,  performance  scores  on  test  trials  were  higher  for  students  trained 
using  ARTT. 

3.  Students  trained  using  ARTT  showed  a  significant  increase  in  performance  scores  when 
transitioned  from  a  difficult  ARTT  task  to  a  more  complex  real-time  task.  Students  trained  in 
real-time  showed  a  significant  decrease  in  performance  when  transitioned  to  a  more  complex 
task. 

4.  ARTT  provided  the  opportunity  for  more  training  events  without  increasing  the  amount  of 
clock  time  spent  in  the  simulator.  ARTT  and  the  additional  training  trials  resulted  in  higher  real¬ 
time  test  scores  than  fewer  training  trials  presented  in  real  time. 
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6.  SUMMARY  AND  DISCUSSION 


The  concept  of  above  real-time  training  was  developed  by  engineers  and  pilots  as  a  practical 
solution  to  an  immediate  problem.  While  early  implementations  of  ARTT  were  promising,  there 
was  no  follow-up  research  by  either  flight-test  engineers  or  cognitive  scientists.  Training 
research  that  is  relevant  to  ARTT  suggests  that  time-compressed  simulation  should  be  most 
effective  for  training  tasks  which  require  significant  time  and  workload  management  and,  for 
trainees  who  have  completed  the  cognitive  portions  of  skill  acquisition.  There  is  no  evidence 
that  ARTT  should  disrupt  or  interfere  with  training  unless:  (a)  the  pace  of  events  overloads  the 
trainee’s  working  memory  or,  (b)  above  real-time  simulation  introduces  artificialities  to  the 
simulation  sufficient  to  produce  negative  transfer  of  training.  There  are  two  proposed  advantages 
to  ARTT.  The  first  is  simple  efficiency.  Using  time-compressed  simulation,  a  pilot  using  ARTT 
can  experience  a  given  number  of  training  events  in  fewer  clock-hours  of  simulator  time  than  a 
pilot  using  conventional,  real-time  simulation.  Alternatively,  the  pilot  using  ARTT  could 
experience  more  training  events  in  a  fixed  amount  of  simulator  time.  The  second  proposed 
advantage  to  ARTT  is  that  ARTT  should  provide  for  easier  transition  to  the  more  demanding 
environment  of  actual  flight  than  normal-time  simulation.  The  experience  of  NASA  test  pilots 
was  that  actual  flight  was  more  demanding  than  simulation;  ARTT  felt  more  like  the  airplane 
than  a  high-fidelity,  real-time  simulation.  Recent  training  research  (see  Guckenberger  et  al, 

1992,  and  Guckenberger  et  al,  1993)  has  supported  both  of  these  hypothesized  advantages  to 
ARTT  for  some  combinations  of  tasks  and  trainees. 

In  the  first  of  three  experiments,  experienced  F- 1 6  pilots  used  either  real  time  or  above  real¬ 
time  training  to  practice  air  combat  tasks.  In  one  task,  the  pilot  used  the  F-16’s  air-to-air  radar  to 
locate  and  track  multiple,  maneuvering  bandit  aircraft.  For  this  task,  training  performance  was 
unaffected  by  ARTT  for  simple  and  moderately  complex  training  scenarios.  Pilot  performance 
was  degraded  by  ARTT  during  training  for  difficult  scenarios.  Pilots  trained  in  real  time  or  using  ' 
ARTT  performed  equally  well  on  test  trials  which  were  presented  in  real  time  for  all  pilots  and 
were  more  complex  than  the  training  trials.  There  was  no  evidence  that  ARTT  provided  for 
improved  transition  to  the  more  demanding  test  trials.  While  performance  of  the  pilots  trained 
using  ARTT  was  degraded  for  some  of  the  training  trials,  there  was  no  evidence  that  ARTT  at 
1 .5-times  real  time  had  a  detrimental  effect  on  test  performance.  Pilot  performance  on  an 
intercept  task  decreased  with  increasing  scenario  complexity  but  was  unaffected  by  ARTT. 

Overall,  the  hypotheses  that  ARTT  represents  an  efficient  training  strategy  was  supported.  Pilots 
trained  using  ARTT  were  able  to  perform  air-combat  tasks  as  well  as  pilots  trained  in  normal 
time  even  though  they  received  only  two  thirds  of  the  simulator  time. 

In  a  second  experiment,  experienced  F- 1 6  pilots  performed  air-combat  tasks  while 
responding  to  in-flight  emergencies.  As  in  the  first  experiment,  pilots  trained  using  ARTT 
performed  a  single  emergency  procedure  and  engaged  a  single  bandit  as  well  as  pilots  trained  in 
real-time  but  no  better.  ARTT  was  more  efficient  than  real-time  training  in  that  pilots  trained 
using  ARTT  were  able  to  perform  on  test  trials  as  well  as  pilots  trained  in  real  time  but  with 
fewer  clock-hours  of  simulator  time.  However,  in  a  more  demanding  multiple  emergency  task, 
pilots  who  received  initial  training  in  real  time  followed  by  additional  practice  using  ARTT 
performed  emergency  procedures  and  engaged  two  bandits  faster  than  pilots  who  received  all 
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training  in  real  time.  This  experiment  supports  the  hypothesis  that  ARTT  may  provide  better 
transition  to  a  more  demanding  task  environment  than  real-time  training.  The  ARTT  pilots  in 
experiment  2  were  able  to  complete  emergency  procedures  more  rapidly  than  pilots  trained  in 
real-time.  Completing  a  given  procedure  more  quickly  will  allow  the  pilot  more  time  for  other 
tasks  and  should  increase  a  pilot’s  effectiveness  in  highly  demanding  conditions. 

In  the  third  experiment,  student  F-16  pilots  performed  the  radar  skills  task  used  in  the  first 
experiment.  Unlike  the  more  experienced  pilots  in  experiment  1,  performance  of  student  pilots 
on  this  task  was  degraded  by  ARTT  for  simple  and  moderately  complex  training  scenarios. 
ARTT  pilot  performance  increased  with  additional  training  trials  but  was  still  reduced  compared 
to  pilots  using  real-time  training.  In  this  experiment,  ARTT  was  used  to  increase  the  number  of 
training  trials  presented  within  approximately  30  minutes  of  simulator  time.  Student  pilots 
trained  in  real  time  received  10  training  trials  while  pilots  trained  using  ARTT  received  15  trials. 
Pilots  trained  using  ARTT  performed  better  on  real-time,  test  trials  than  pilots  trained  in  real 
time.  The  combination  of  ARTT  plus  additional  training  trials  led  to  improved  performance  on 
the  test  trials  without  increasing  clock-hours  in  the  simulator. 

Overall,  the  hypothesis  that  ARTT  is  more  efficient  than  real-time  simulation  was  supported. 
These  results  concur  with  the  findings  of  Schneider  et  al.  (1982)  and  Vidulich  et  al.  (1983). 
Compared  to  real-time  simulator  training,  time-compressed  training  that  does  not  overload  the 
trainee’s  working  memory  can  support  equivalent  levels  of  test  performance  with  fewer  hours  of 
clock-time  in  a  simulator  or  improved  test  performance  with  equal  amounts  of  clock-time  in  a 
simulator.  The  hypothesis  that  transfer  of  training  to  a  more  demanding  task  will  be  increased  by 
ARTT  without  additional  training  trials  was  also  supported  for  some  tasks.  Pilots  who  received 
initial  training  in  real  time  followed  by  additional  practice  using  ARTT  performed  faster  than 
pilots  trained  in  real  time  for  highly  demanding  tasks. 
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7.  CONCLUSIONS  AND  RECOMMENDATIONS 


1 .  The  Air  Force  Human  Systems  Center’s  Technical  Planning  Integrated  Product  Team  (HSC 
TPIPT)  has  proposed  that  ARTT  may  benefit  Air  Force  training  by: 

(a)  Increasing  task  performance, 

(b)  Increasing  trainee  retention  of  skills, 

(c)  Increased  situation  awareness, 

(d)  Decreasing  real-time  workload, 

(e)  Decreasing  real-time  stress, 

(f)  Increasing  the  rate  of  skill  acquisition  (faster  and  steeper  learning  curves), 

(f)  Reduced  simulator  and  aircraft  training  time,  and 

(g)  More  effective  emergency  procedures  training 

The  scope  of  the  present  effort  limited  research  efforts  to  focus  on  items  1  (task  performance),  6 
(skill  acquisition),  7  (training  time),  and  8  (emergency  procedures).  Results  from  the  present 
study  support  the  following  conclusions: 

•  Task  performance  under  Above  Real-Time  Training  is  degraded  compared  to  real-time 
training.  However,  real-time  test  performance  was  as  good  or  better  for  pilots  trained  using 
ARTT  than  for  pilots  trained  using  real-time  simulation. 

•  There  is  a  trade-off  between  skill  acquisition  and  training  time.  ARTT  which  does  not 
overload  the  trainee’s  working  memoiy  can  provide  equivalent  training  with  fewer  clock-hours 
in  a  simulator  or  better  training  with  equal  time  in  a  simulator  compared  to  real-time  training. 

•  The  results  of  this  study  support  hypothesis  that  ARTT  benefits  training  emergency 
procedures  under  some  conditions.  Practice  using  ARTT  after  initial  training  increased  pilots’ 
response  speed  when  performing  emergency  procedures. 

2.  As  an  instructional  strategy,  ARTT  is  inexpensive  to  implement  and  can  increase  time 
efficiency  for  many  training  tasks.  The  major  difficulty  introduced  by  ARTT  is  that  training 
performance  is  degraded  compared  to  real-time  simulation.  If  pilots  are  required  to  meet  a 
minimum  standard  of  task  proficiency  before  progressing  to  the  aircraft  or  to  another  task,  ARTT 
will  increase  training  systems  management  complexity.  Training  managers  must  establish 
alternative  proficiency  criteria  for  ARTT  or  allow  for  additional  training  trials  to  overcome  the 
greater  difficulty  of  ARTT.  Since  ARTT  is  more  time  efficient,  these  additional  trials  may  not 
require  increased  clock-time  in  the  simulator. 
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3.  ARTT  combined  with  real-time  training  decreased  pilot’s  emergency  response  time.  Further 
research  is  recommended  to  determine  what  combinations  of  tasks,  trainees,  and  training 
procedures  will  most  benefit  from  ARTT.  Further  research  is  also  recommended  regarding  skill 
retention  and  the  duration  of  ARTT  effects. 

4.  The  use  of  individual,  low-cost  training  devices  is  becoming  more  common  within  the  Air 
Force  particularly  for  continuation  training.  The  F-15  weapons  and  tactics  trainer  (WTT),  F-16 
unit  training  device  (UTD),  and  F-16  multi-task  trainer  (MTT)  are  examples  of  this  family  of 
training  systems.  ARTT  combined  with  concentrated  practice  on  specific  tasks  such  as  the  radar 
skills  task  in  the  present  study  may  be  a  useful  training  strategy  to  maintain  proficiency  or 
prepare  for  an  exercise.  Further  research  on  training  strategies  which  will  maximize  training 
benefits  of  individual,  low-cost  training  devices  is  recommended. 

5.  ARTT  is  only  one  application  of  variable  time  training.  The  hyper-time  algorithm  described 
in  section  2.4  above  allows  for  rapid  adjustment  of  simulator  time.  While  simulator  time  would 
normally  be  set  equal  to  clock-time  for  most  training,  it  could  also  be  set  to  less  than  real  time  for 
novices  and  to  greater  than  real-time  as  a  final  step  in  training.  Further  research  on  the  use  of 
variable  time  training  is  recommended. 
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NASA/DFRC  memo,  12  Apr  73,  Documentation  of  a  Simulator  Study  of  an  Altered  Time  Base, 

which  describes  fast-time  training  concept. 


Appendix  A  is  a  transcription  of  Jack  L.  Kolf  s  memo  describing  the  application  of  ARTT  to 
the  M2-F3  lifting  body  program. 
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NATIONAL  AERONAUTICS  AND  SPACE  ADMINISTRATION 
FLIGHT  RESEARCH  CENTER 
Edwards,  California  95523 


April  12,  1973 

MEMORANDUM 

TO:  Director  of  Research 

FROM:  X-25B  Project  Manager 

SUBJECT:  Documentation  of  a  Simulator  Study  of  an 

Altered  Time  Base 


For  at  least  the  last  ten  years  a  study  of  pilot  comments 
following  rocket  aircraft  research  flights  has  shown  one 
recurring  problem.  That  is,  regardless  of  type  or  amount  of  pre¬ 
flight  simulator  training  accomplished  by  the  pilot,  the  actual 
flight  appears  to  take  place  in  a  much  faster  time  frame  than 
real  time.  This  problem  is  much  more  apparent  in  a  pilot's  first 
few  flights  in  any  given  aircraft  and  seems  to  lessen  almost 
linearly  with  experience.  At  any  rate,  no  pilot  has  ever 
obtained  the  necessary  experience  in  any  particular  rocket 
research  aircraft  to  completely  slow  the  actual  flight  reference 
down  to  a  real  time  environment. 

It  is  apparent,  and  intuitive,  that  this  problem  has  to 
compromise  both  the  quantity  and  quality  of  research  data 
gathered  on  a  particular  flight  as  well  as  increasing  the  pilot's 
workload  and  response  effectiveness  in  the  event  of  an  emergency 
situation . 

Therefore,  it  was  decided  to  attempt  an  experiment  during  the  M2- 
F3  flight  program  designed  to  at  least  minimize  the  effects  of 
this  speeded  up  time  reference.  The  intent  was  to  train  an 
experienced  M2  pilot  in  the  simulator  using  a  speeded  up  time 
reference  and  carry  the  training  right  up  to  an  actual  flight. 
This  would  allow  a  comparison  of  perceived  workload  on  a  flight 
trained  for  in  this  manner  and  a  similar  one  conducted  in  the 
usual  method.  Unfortunately  the  compressed  schedule  necessitated 
by  the  premature  demise  of  the  M2-F3  did  not  allow  any  simulator 
time  for  program  changes.  As  a  second  choice,  we  decided  to 
accomplish  the  simulator  study  using  a  flight  that  had  already 
been  flown.  Although  comments  based  on  a  flight  flown  months  in 
the  past  are  not  extremely  reliable,  some  rather  significant  and 
enthusiastic  response  and  results  were  obtained. 

The  actual  changes  made  to  the  simulator  turned  out  to  be  much 
less  extensive  than  anticipated.  It  was  determined  that  only  the 
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scale  had  to  be  changed  on  the  analog  integrators  and  the 
integration  rate  on  the  digital  computer.  This  amounted  to:  1) 

4  cards  or  steps  were  added  to  the  XDS-9300  digital  program,  2) 

19  pot  setting  changes  were  needed  on  the  EAI-231R  main  analog 
computer,  3)  13  pot  setting  changes  on  the  EAI-TR48  OAS  control 
system  analog  computer,  and  4)  1  pot  setting  change  on  the  EAI- 
TR5  pitch  stick  trim  rate  analog  computer.  After  having  made 
this  switch  a  few  times,  it  was  found  that  the  entire  change  from 
real  time  to  the  speeded  up  time  or  vice  versa  could  be 
accomplished  in  less  t6han  fifteen  minutes. 

The  only  other  change  required  was  more  complex  but  only  had  to 
be  made  once.  One  of  the  prime  instruments  used  by  a  pilot 
during  boost  is  the  engine  timer.  He  uses  this  for  both  profile 
control  and  data  maneuvers  as  well  as  an  energy  management  tool. 
This  timer,  then,  had  to  be  altered  to  reflect  the  same  time  base 
as  the  basic  simulation.  For  the  optimum  results,  a  new  watch 
should  have  been  modified  to  complete  one  sweep  (100  sec  on  the 
dial)  in  the  same  ratio  as  the  speeded  up  time  base.  In  order  to 
save  time  it  was  decided  to  just  have  a  new  face  made  that  showed 
one  sweep  to  be  150  seconds.  This  established  the  time  base  for 
the  rest  of  the  simulation  as  1.5  times  real  time.  Pilot 
comments  indicated  that  the  difference  between  this  timer  and  the 
100  second  one  in  the  airplane  did  not  compromise  the  study 
results.  Also,  the  1.5  time  factor  appeared  to  be  near  optimum 
for  the  M2-F3. 

All  three  of  the  current  M2-F3  pilots  were  asked  to  fly  the 
flight  M-42-57  profile  until  they  regained  enough  proficiency  to 
retain  the  validity  of  the  study.  They  were  then  asked  to  fly 
the  same  profile  with  the  speeded  up  time  base.  Less  than  30 
seconds  were  required  to  evoke  comments  on  the  realistic 
appearance  of  this  simulation  to  an  actual  flight.  In  no  case 
was  more  than  one  run  necessary  to  establish  the  fact  that  we  had 
"stumbled"  on  an  effective  research  and/or  pilot  training  tool. 
The  study,  although  shorter  and  less  detailed  than  desired,  was 
conclusive  enough  to  convince  us  to  make  use  of  the  technique  in 
the  X-24B  program  if  a  similar  change  is  feasible  on  the  air 
force  simulation  computers. 

The  following  conclusions  and/or  observations  are  offered  for 
information : 

1)  This  technique  could  find  application  wherever  procedural 
trainers  are  used.  Possibilities  range  from  airline  operations 
to  highly  complex  research  air/space  craft. 

2)  Further  study  would  be  required  to  optimize  the  time  base 
factor.  It  would  appear  to  be  a  function  of  aircraft. 
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individual,  task,  and  experience.  A  precise  factor  in  each  case 
may  or  may  not  even  be  necessary. 

3)  In  a  program  such  as  the  lifting  body  program,  probably  only 
the  last  hour  or  two  of  training  would  be  in  the  fast  time  base. 
Preliminary  training  would  be  accomplished  as  it  is  now . 

4)  Overall  program  results  would  most  likely  be  improved  if 
second  level  positions  such  as  "NASA-1"  flight  controller  and  the 
flight  planner  trained  in  the  same  speeded  up  time  base  along 
with  the  pilot. 

5)  An  electric  engine  timer  (or  clock  in  other  applications) 
would  allow  the  time  base  to  be  varied  at  will  without  the 
necessity  of  a  time  consuming  gear  change  in  the  timing 
mechanism. 

6)  Emergency  or  non-standard  procedural  techniques  could 
benefit  highly  from  this  system  of  training  as  timing  is  usually 
most  critical  in  such  cases . 

Finally,  we  feel  this  technique  could  and  should  be  evaluated  in 
any  program  using  a  simulator.  As  mentioned,  the  X-24B  project 
office  already  is  planning  to  if  time  and  equipment  allow. 


( signed) 

Jack  L.  Kolf 
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Hoey,  R.G.  (1976).  “Time  compression  as  a  means  for  improving  value  of  training  simulators”. 
Unpublished  paper  on  time-compressed  training  for  Dr  Van  Slyke,  Course  SSM  517. 


Robert  G.  Hoey,  a  test  engineer  at  NASA’s  Dryden  Flight  Research  Center,  documented  the 
experiences  of  flight  test  pilots  using  Above  Real-Time  Training  (ARTT).  This  paper  was 
written  while  Mr.  Hoey  was  pursuing  graduate  studies.  The  following  is  a  transcription  of  Mr. 
Hoey’s  typewritten  manuscript. 
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INTRODUCTION 

In  a  paper  discussing  discrepancies  between  flight 
simulators  and  the  actual  flight  experience,  Milton  0.  Thompson, 
X-15  rocket  airplane  pilot,  concluded  that  "The  actual  flight  of 
environment  must  still  be  investigated,  since  the  effects  of 
apprehension  and  anxiety  on  the  pilot  cannot  yet  be  simulated".1 
Recent  experiments  performed  at  the  National  Aeronautics  and 
Space  Administration  Dryden  Flight  Research  Canter  (NASA  DFRC) 
indicate  that  a  psychological  effect  similar  to  that  produced  by 
anxiety  and  stress  may  be  produced  by  operating  the  pilot 
training  simulators  raster  than  real  time  (time-compression) . 
Although  the  experimental  work  leading  to  the  idea  of  time 
compression  training  was  not  conducted  under  controlled, 
laboratory-type  conditions,  the  concept  has  been  applied  quite 
successfully  to  a  recent  fight  test  program.  This  paper  discusses 
the  background  of  the  time-compression  concept,  some  aspects  Of 
the  psychological  basis  for  its  application,  an  evaluation  of  the 
experimental  results  and  some  ideas  on  potential  application. 

BACKGROUND 


Research  Airplane  Piloting  Task 

The  piloting  task  associated  with  the  flight  testing  of 


Thompson,  Milton  0.,  "General  Review  of  Piloting  Problems 
Encountered  During  Simulation  and  Flights  of  the  X-15",  9th 
Annual  Report  to  the  Aerospace  Profession,  Society  Of 
Experimental  Test  Pilots  Annual  Symposium,  1965 
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research  aircraft  has  typically  been  highly  specialized  and 
demanding.  As  a  result  of  the  relatively  unknown  environment  of 
these  aircraft  and  the  high  cost  and  safety  implications  of  each 
flight,  it  was  probably  natural  that  sophisticated,  real  time 
flight  simulators  would  find  their  first  practical  application  in 
this  field.  Flight  simulators  were  first  used  in  support  of  the 
X-2  rocket  powered  research  airplane  in  approximately  1955, 
gaining  acceptance  rapidly  thereafter.  A  highly  sophisticated 
simulation  of  the  famous  X-15  research  aircraft  was  used 
successfully  throughout  its  test  program.2  It  is  estimated  that 
each  X-15  pilot  averaged  more  than  1_0  hours  in  the  X-15  simulator 
for  every  10  minutes  that  he  spent  in  actual  flight. 

For  research  aircraft  test  programs  the  simulator  is  used 
first  to  plan  each  flight  in  minute  detail.  Typical  air-launched, 
rocket-powered  fights  are  between  7  and  11  minutes  in  total 
duration.  The  short  duration  of  each  flight  dictates  that  the 
pilot  commit  to  memory  all  of  the  necessary  flight  plan  details, 
test  maneuvers,  and  emergency  procedures.  The  simulator  is  then 
used  to  train  the  pilot  to  fly  the  normal  mission,  and  to 
recognize,  and  respond  properly,  to  all  conceivable  emergencies 
or  unexpected  flight  occurrences. 

Although  the  flight  simulator  has  proven  to  be  an  extremely 
valuable  contribution  to  the  successful  and  safe  flight  testing 
of  research  aircraft,  the  pilots  have  continually  expressed 
opinions  such  as;  "It  sure  seems  to  happen  faster  in  the  real 

2  Hoey,  R.G.,  Day,  R.E.,  "Mission  Planning  and  Operational 
Procedures  for  the  X-15  Airplane",  NASA  TN  D-1159,  March  1962 
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airplane",  or  "I  had  the  feeling  that  I  was  'behind  the 
airplane'"  or  "I  felt  rushed  trying  to  get  everything  done 
compared  to  the  simulator".  These  comments  were  generally  felt  to 
be  related  to  the  differences  in  the  physiological  environment. 
Certainly  the  visual  cues,  motion  cues  (including  relatively 
large  "g"  values),  pressure  suits,  and  other  obvious  physical 
environmental  differences  between  the  simulator  and  actual 
flight,  could  not  be  discounted.  (All  simulators  referred  to  in 
this  paper  were  fixed  base  simulators) . 

Early  Simulator  Time-Compression  Experiments 

The  similarity  of  each  pilots  subjective  evaluation  of  the 
simulator/flight  comparison  led  to  a  brief  and  informal 
experiment  in  1971.  After  completion  of  the  M2-F3  Lifting  Body 
flight  test  program,  but  before  dismantling  the  simulator,  the 
engineers  modified  the  computers  in  the  simulator  so  that  they 
could  be  operated  at  a  selectable  time  rate  which  was  faster  than 
real  time.  (This  was  quite  easily  accomplished  by  applying  the 
same  speed-up  factor  to  the  integration  rate  of  all  integrators 
in  the  computer) .  Pilots  who  had  previously  flown  the  actual  M2- 
F3  evaluated  the  simulator  with  various  time-compression  factors. 
The  pilots  selected  a  factor  of  about  1H  times  faster  than  real 
time  as  the  factor  which  most  represented  the  way  it  felt  in 
flight.  They  were  unanimous  and  highly  enthusiastic  about  the 
realism  provided  by  the  1H  time-compression  simulation.  Again  the 
validity  of  the  concept  was  largely  discounted  by  the  technical 
community  due  to  the  known  large  environmental  differences 
between  the  fixed  base  simulator  and  fight. 
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Application  To  Remotely  Piloted  Research  Vehicles  (RPRV's) 

In  1972  engineers  at  NASA  DFRC  conceived  of  the  idea  of 
expanding  upon  the  radio-control-model-airplane  technology  in 
order  to  conduct  hazardous  flight  tests  without  exposing  the 
pilot  to  the  associated  dangers.  The  concept  was  to  place  the 
pilot  in  a  simulator-like  cockpit  on  the  ground  with  typical 
aircraft-type  controls  and  displays.  These  controls  and  displays 
were  linked  to  the  actual,  unmanned,  flight  vehicle  through  radio 
signals.  An  "uplink"  transmitted  pilot  commands  to  the  individual 
control  surfaces  on  the  flight  vehicle.  Instrumentation  on  the 
vehicle  measured  the  aircraft  responses  and  appropriate  data  were 
transmitted  to  the  ground  via  a  "downlink" .  Vehicle  response  was 
presented  to  the  pilot  on  his  ground  cockpit  displays  in  real 
time.  This  permitted  the  pilot  to  close  the  loop  in  exactly  the 
same  fashion  as  he  would  have,  had  he  been  flying  on  instruments 
in  the  flight  vehicle,  thus  the  name.  Remotely  Piloted  Research 
Vehicle  (RPRV) .  The  effort  culminated  in  the  flight  testing  Of  a 
subscale  glider  version  Of  the  F— 15  air  superiority  fighter.3 

A  flight  simulator  was  again  a  key  factor  in  the  successful 
accomplishment  of  this  program,  and,  although  the  pilot  remained 
on  the  ground,  the  piloting  task,  procedures  and  training 
requirements  were  identical  to  those  discussed  earlier  for 
piloted  research  aircraft. 

Early  in  the  F-15  RPRV  program  the  simulator  was  programmed 


3  Holzeman,  Euclid  C.  (Ed)  "Initial  Results  From  Flight  Testing  a 
Large,  Remotely  Piloted  Airplane  Model",  NASA  TM  X-56024,  March 
1974 
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to  operate  in  either  real  time  or  1.4  times  real  time.4 
Throughout  the  program  the  1.4  time-compression  simulation  was 
used  for  final  pilot  training  before  a  fight. 

Implications  Of  RPRV  Application 

As  in  the  M2-F3  simulator  experiment,  the  F-15  RPRV  pilots 
felt  that  the  1.4  time-compression  was  a  realistic  simulation  of 
the  way  it  felt  during  the  actual  flight.  There  is  an  important 
difference,  however,  between  the  M2-F3  application  and  the  F-15 
RPRV  application  of  time-compression.  For  an  RPRV  the  pilot's 
physical  environment  while  flying  the  simulator  is  essentially 
identical  to  his  environment  while  flying  the  actual  vehicle!  He 
is  in  no  immediate  danger,  has  no  motion  or  additional  visual 
cues  present,  and  is  in  a  comfortable,  shirtsleeve  environment. 
Any  differences  in  the  perception  of  time  passage  must  therefore 
be  related  primarily  to  psychological  effects. 

Physiological  Data  Base 

Man's  first  real  exposure  to  the  zero-g  environment  occurred 
during  the  X-15  program.  This  prompted  the  development  of  a 
sophisticated  biomedical  monitoring  package  for  measuring  the 
physiological  status  of  the  pilot  during  flight.  The  most  useful 
measurement  was  found  to  be  an  EKG.  The  medical  monitoring 
program  continued  throughout  the  X-15  program  and  persisted 

4  The  1.4  time-compression  factor  was  not  related  to  the  scaling 
effects  between  the  subscale  F-15  model  and  the  full  size  F-15. 
The  1.4  time-compression  factor  was  applied  over  and  above  the 
other  mathematically-correct  scaling  factors  which  correct  for 
differences  in  size  and  inertia. 
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through  the  entire  lifting  body  flight  test  program.  During  this 
program  the  data  base  was  expanded  to  include  EKG  monitoring  of 
ground  control  personnel  as  well  as  the  pilots  in  flight.  (Test 
pilots  served  as  ground  controllers  in  the  control  room,  thus 
control  room  data  was  seen  as  part  of  the  data  base  for  each 
individual  test  pilot) . 

Over  the  years  a  considerable  volume  of  data  has  been 
accumulated  with  some  surprising  results .  A  repeatable  pattern  of 
pilot's  heart  rate  variation  during  a  fight  was  found  which 
correlated  with  key  flight  events.5  Three  peaks  were  evident  in 
the  pilot's  heart  rate.  The  first  peak  occurred  at  launch,  which 
was  the  beginning  of  his  term  of  responsibility.  The  second  peak 
occurred  at  rocket  engine  burnout  where  the  pilot's  task  changed 
from  trajectory  control  to  the  performing  of  test  maneuvers.  The 
third  and  usually  highest  peak  occurred  at  landing  which  marked 
the  end  of  the  experience. 

The  only  significant  variation  on  this  pattern  was  the 
expected  increase  in  heart  rate  associated  with  sustained,  high  g 
maneuvers  such  as  during  reentry.  Although  the  maximum  values  of 
heart  rate  varied  considerably  between  pilots,  the  pattern 
relating  peak  values  with  the  same  key  fight  events  was 
consistent  for  all  pilots. 

Heart  rate  data  on  ground  controllers  also  provided  some 
surprises.  Peak  values  were  nearly  as  high  when  a  pilot  was 

5  Bratt,  Harry  R. ,  Lt.  Col.  USAF,  "Biomedical  Aspects  Of  the  X-15 
Program,  1959-1964",  AFMC-TR-65-24 ,  August  1965 
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serving  as  a  ground  controller  as  they  were  then  he  was  actually 
flying  the  aircraft.  The  highest  rate  usually  occurred  at  launch 
which  corresponded  with  the  end  of  the  ground  controllers  term  of 
primary  responsibility.  (He  transmitted  the  final  decision  to 
launch) . 

Heart  rates  measured  during  simulator  training  runs  were 
quite  low  and  exhibited  no  particular  patterns  since  the  pilots 
were  not  experiencing  any  stress. 

The  biomedical  monitoring  program  was  extended  to  gather 
data  on  the  pilots  and  ground  controllers  in  the  F-15  RPRV 
program  even  though  both  individuals  remained  in  a  ground 
environment.  In  general,  the  same  patterns  and  same  peak  values 
of  heart  rate  were  observed  for  the  RPRV  pilots  while  remotely 
flying  the  flight  vehicles,  as  had  occurred  when  these  same 
pilots  were  actually  flying  in  the  test  aircraft. 

The  observed  physiological  and  postulated  psychological 
stress  on  test  pilots  is  summarized  in  Table  I  for  the  research 
airplane  test  programs  discussed  earlier. 

(Note:  Comments  in  this  section  represent  rather  gross 
generalizations  of  a  great  volume  of  EKG  data,  much  of  which  is 
unpublished  at  this  time.  The  general  trends  reported  herein  were 
obtained  by  interviews  with  the  pilots  and  biomedical  personnel 
at  NASA  DFRC  who  are  actively  engaged  in  the  data  gathering 
process) . 
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PSYCHOLOGICAL  BASIS 

Perception  of  Time 

We  are  all  aware  of  an  "internal  sense  of  time"  which  is  often 
different  from  that  recorded  by  clocks  and  calendars.  Hoaglund 
performed  experiments  which  related  the  subjective  sense  of  time 
to  body  temperature.  (According  to  Hoaglund's  results  a. time- 
compression  factor  of  1.4  would  be  produced  by  a  4.5  degree  F 
increase  in  body  temperature) .  As  related  by  Cohen6  "Hoaglund 
thereafter  explored  the  subject  extensively  and  concluded  that 
there  are  chemical  pacemakers  in  the  brain  that  govern  the  speed 
of  its  metabolism  and  thereby  affect  the  rhythm  of  subjective 
time”.  Certainly  this  statement  is  consistent  with  recent 
findings  in  the  rapidly  expanding  field  of  biochemistry.  The 
question  relative  to  this  topic  is;  what  are  the  physiological 
and  psychological  factors  which  trigger  these  chemical 
pacemakers,  how  are  they  related,  and  how  can  we  measure  their 
effects? 

Cohen  discusses  evidence  of  "an  interrelation  of  inner  clocks 
and  sensory-motor  activity.  Each  can  influence  the  other".  The 
pilot  training  functions  described  earlier  involve  both  sensory- 
motor  training  (operation  of  controls)  and  mental  training 
(memory  and  decision  criteria) .  A  change  to  the  inner  clock 
produced  by  one  or  more  psychological  factors  could  very  well 
introduce  an  imbalance  or  difference  between  the  mental 


6  Cohen,  John,  "Psychological  Time",  Scientific  American,  Nov. 
1964,  pp.  116-124 
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processes  and  the  sensory  motor  processes.  This  might  be  manifest 
to  the  pilot  as  a  feeling  of  being  "rushed"  mentally. 

Measurement  of  Stress 

Stress  is  defined  by  James  G.  Miller  as  a  "force  that  pushes 
the  functioning  of  important  (body)  subsystems  beyond  their 
ability  to  restore  equilibrium  through  ordinary,  non-emergency, 
adjustment  processes".7  It  has  long  been  recognized  that  stress 
can  be  produced  by  both  physiological  and  psychological 
stressors;  however,  the  only  known  yardsticks  for  measuring 
stress  are  physiological  responses.  There  has  been  little  solid 
laboratory  or  experimental  work  on  the  isolation  of  psychological 
stress  from  physiological  stress.8  It  appears  that  our  present 
knowledge  is  such  that  there  are  recognized  physiological 
measures  of  stress  which  do,  in  fact,  measure  stress  regardless 
of  the  type  of  stressor  present.  In  his  discussion  of 
psychological  time  Cohen  relates;  "Ones  orientation  toward  future 
events  is  often  characterized  by  a  'gradient  of  tension'.  The 
heart  beats  faster  as  the  clock  emphasizes  that  a  fateful  moment 
draws  near."9  Clearly  this  common  phenomenon,  with  which  we  can 

Miller,  James  G.,  "A  Theoretical  Review  Of  Individual  and  Group 
Psychological  Reactions  to  Stress",  In  Grosser,  Wechsler, 
Greenblatt  (Ed)  The  Threat  of  Impending  Disaster,  M.I.T.  Press, 
1964,  p . 13 

8  Lazarus,  Richard  S.,  "A  Laboratory  Approach  to  the  Dynamics  of 
Psychological  Stress",  American  Psychologist,  1964,  19,  pp.  400  - 
411 

9  Cohen,  John,  "Psychological  Time:,  Scientific  American,  Nov. 

1964,  pp.  116  -  124  '  ““ 
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all  identity,  is  a  form  of  psychological  stress  characterized  by 
a  physiological  change,  namely,  increased  heart  rate. 

Heart  rate  has  often  been  used  as  an  indicator  of  stress 
level  and  for  the  remainder  of  this  paper  it  will  be  assumed  that 
it  is  a  valid  measure.  This  measuring  device  is  of  particular 
interest  in  this  case  since  a  considerable  volume  of  EKG  data  has 
been  accumulated  during  past  research  airplane  test  programs. 
Fear,  Anxiety,  Responsibility 

If  we  assume  for  a  moment  that  the  physiological  environment 
of  the  RPRV  pilot  is  constant,  regardless  of  whether  he  is  flying 
the  actual  flight  vehicle  or  a  computer  simulation,  we  must 
conclude  that  any  difference  in  measured  heart  rate  between  the 
two  is  attributable  to  psychological  stress.  (Table  I)  But  what 
kind  of  psychological  stress? 

Fischer  10  differentiates  between  anxiety  and  fear  as 
follows : 

"Being  Anxious  -  uncertain  actualization  of  a  lived 
for-and-toward  world  and  to-be-realized  identity. 

Being  Fearful  -  uncertain  defense  of,  and  holding  onto, 
that  which  I  already  am  and  have. 

Being  anxious  implies  some  task-related  impotence  and 
apprehending  of  fulfillment.  Being  fearful  implies  defense 
mechanisms  and  not  impotence." 


10  Fischer,  William,  Theories  of  Anxiety,  Harper  and  Row,  1970, 

p .  166 
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A  further  differentiation  between  fear  and  anxiety  relates 
to  the  time  element.  Anxiety  is  based  on  the  subjects  projection 
of  future  events  which  have  not  yet  occurred,  whereas  fear  is 
based  on  recognition  of  some  event  which  has  immediate 
connotations  of  danger. 

A  test  pilot  actually  flying  in  a  research  aircraft  might 
reasonably  be  expected  to  experience  some  elements  of  both 
anxiety  and  fear  depending  on  how  he  views  his  assigned  tasks  and 
his  relative  safety.  Notice  that  he  is  also  experiencing 
physiological  stressors  due  to  the  visual  and  motion  stimuli. 

The  RPRV  pilot,  in  addition  to  not  experiencing  any 
physiological  stressors,  is  in  absolutely  no  danger  and  should 
therefore  not  experience  any  significant  element  of  personal 
fear.  High  pilot  heart  rates  (and  therefore  assumed  stress 
levels)  were  observed  regardless  of  whether  the  pilot  was 
actually  inside  a  test  aircraft  or  flying  it  from  a  ground 
cockpit.  This  evidence  leads  us  to  conclude  that  the  primary 
stressor  producing  the  high  heart  rate  is  anxiety,  and  that 
physiological  stressors  and  psychological  fear  play  a  relatively 
small  role.  This  is  also  consistent  with  the  previously  mentioned 
correlation  of  peak  heart  rates  with  responsibility-related 
flight  events. 

In  assessing  anxiety  under  various  conditions,  Fischer  11 
found  several  "striking  structural  similarities"; 

"(1)  In  each - we  discovered  a  situational  event  that 


Ibid.,  pp.  129  -  130 
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constituted  the  focus  of  the  individual's  anxious  orientation. 

(2)  In  each - was  also  the  question  of  the  individual's 

identity - in  some  manner,  an  expression  of  the  person's  world. 

(3)  - the  network  of  relations  and  projects - of  both  his 

world  and  his  identity,  emerqe  as  demanding  to  be  sustained .  This 

demanding - is  experienced  as  a  question  of  necessity,  an 

absolute  requirement  of  life.  There  are  no  conceivable 
alternatives - . 

(4)  - the  question  of  ability  -  specifically,  the  personal 

sense  of  uncertain  ability  or  competence  -  is  essential  to  the 
experience  of  anxiety." 

These  characteristics  appear  to  fit  quite  well  into  the  test 
pilots  situation  prior  to  and  during  a  test  flight  regardless  of 
whether  he  is  inside  the  test  vehicle  or  flying  it  from  the 
ground.  In  either  case  he  is  totally  responsible  for  the  success 
or  failure  of  the  fight  and  all  that  it  might  portend  for  his 
personal  future.  The  effect  of  this  sense  of  piloting  respon¬ 
sibilities  was  graphically  demonstrated  by  Dr.  Roman.12  Two  EKG- 
instrumented  test  pilots  flew  a  very  demanding,  high  speed,  low 
altitude  flying  task  in  a  two-place  jet  aircraft.  On  each  run, 
one  of  the  pilots  was  actually  flying  the  aircraft  as  "pilot-in¬ 
command",  and  the  other  was  a  "passenger".  Each  pilot  made 
several  runs  in  each  role.  Although  both  pilots  were  experiencing 
the  same  physiological  stressors,  and  psychological  fears  (if 
any)  the  "pilot-in-command"  consistently  registered  much 

12  Roman,  James,  M.D.,  "Risk  and  Responsibility  as  Factors 
Affecting  Heart  Rate  in  Test  Pilots  -  The  Flight  Research  Program 
II",  Aerospace  Medicine,  Vol.  36  no.  6,  June  1965 
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higher  relative  heart  rates  than  his  "passenger"  (relative  to 
each  individual's  base  line  heart  rate) . 

Learning  Methods 

The  applicability  of  a  pilot-in-the-loop,  real  time 
simulator  to  the  pilot  training  task  has  been  widely  accepted  and 
need  not  be  dealt  with  in  any  depth.  It  represents  the  epitome  of 
reinforcement  and  rapid  feedback  learning  methods.  As  indicated 
earlier  pilot  training  in  a  simulator  includes  both  perceptual 
motor  skills  and  memory  and  decision-making  skills.  In  a  task 
analysis  of  perceptual  motor  skills  Fitts13  described  three 
phases  of  the  learning  process; 

(1)  Cognitive  Phase  -  Transfer  of  previous  training  is  most 
important  -  getting  the  "feel"  of  some  situation. 

(2)  Fixation  Phase  -  Correct  patterns  of  motor  action  are 
refined  and  fixated.  Longest  and  most  difficult  phase. 

(3)  Automation  phase  -  Rapid,  automatic  performance.  Errors 
at  a  minimum.  "Skill  becomes  not  only  well  integrated,  but 
resistant  to  the  effects  of  stress  and  interference  from  other 
concurrent  activities"  . 

Simulator  pilot  training  of  the  kind  described  earlier 


Fitts,  P.M. , (1962)  "Factors  in  Complex  Skill  Training",  In  R. 
Closer  (Ed)  Training.  Research  and  Education,  Pittsburgh,  Pa., 
Univ.  of  Pittsburgh  Press 

and 

Fitts,  P.M. ,(1964)  "Perceptual-Motor  Skill  Learning",  In  A.W. 
Melton  (Ed) ,  Categories  of  Human  Learning,  New  York:  Academic 
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probably  achieves  the  "fixation  phase"  for  all  required  motor 
skills  with  some  skills  attaining  the  "automation  phase”.  Some  of 
the  learned  motor  skills  obviously  pass  beyond  the  psychological 
refractory  phase  (wherein  short  term  memory  acts  in  a  serial 
manner,  processing  one  input  at  a  time)  and  reach  the  automatic 
phase  (wherein  multi  stimulus  inputs  can  be  handled) . 

During  all  of  the  research  airplane  test  programs  it  can  be 
generally  stated  that  the  pilots  performed  their  tasks 
exceptionally  well,  in  spite  of  the  professed  feeling  of  being 
"rushed"  compared  with  the  simulator.  It  would  appear  that  the 
motor  skills  learned  on  the  simulator  became  reflex  actions  for 
the  most  part;  therefore,  the  proper  pilot  actions  did  occur  at 
the  proper  times.  The  stress  produced  by  anxiety  resulted  in  an 
alteration  to  the  pilot's  internal  sense  of  time  which,  in  turn, 
produced  an  inconsistency  between  his  physical  and  mental 
responses.  Pilots  described  their  awareness  of  this  inconsistency 
as  a  "feeling  of  being  rushed".  The  f aster-than-real-time 
simulation  artificially  produced  a  mental  situation  similar  to 
that  produced  by  anxiety.  The  pilots  have  recognized  and 
correlated  this  effect  subjectively  as  a  result  of  alternately 
flying  the  simulator  (without  anxiety  but  with  1.4  time- 
compression)  ,  then  the  actual  flight  vehicle  (with  anxiety  but  in 
real  time) . 

EVALUATION  AND  POTENTIAL  APPIICATION 
Task  Dependencies 

The  application  of  a  time-compression  factor  of  1.4  has 
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been  shown  to  be  beneficial  (at  least  subjectively)  for  the 
particular  task  of  training  for  F-15  RPRV  flights.  This  task  is 
typical  of  nearly  all  research  aircraft  flying  tasks  which  are 
characterized  by  very  high  pilot  work  loads  for  relatively  short 
time  periods  (7-11  minutes) .  Application  to  longer  or  shorter 
time  periods  and/or  lower  or  higher  work  loads  is  unknown  at  this 
time.  It  is  possible  that  the  optimum  time-compression  factor  is 
related  in  some  manner  to  pilot  work  load. 

It  is  most  interesting  to  notice  the  implication  in  the 
previous  discussion  that  a  persons  inner  sense  of  time  might  be 
related  directly  to  heart  rate.  If  such  a  relationship  could  be 
established,  observed  pilot  heart  rate  variations  while 
accomplishing  a  particular  real-world  task  (such  as  landing  an 
airliner)  could  be  used  to  program  a  variable  time-compression 
factor  for  a  training  simulator.  In-flight  emergency  situations 
might  entail  sudden  changes  in  stress  level  (as  might  be 
associated  with  fear  rather  than  anxiety)  thus  requiring  some 
different  logic  for  changing  the  time-compression  factor  of  the 
simulator . 

Physical  Inconsistencies 

Although  the  affect  of  anxiety  on  the  mental  processes  may 
be  approximately  simulated  by  time-compression,  the  resulting 
speed-up  of  the  physical  task  is  probably  incorrect.  The  muscular 
effort  required  to  reach  and  activate  a  switch  or  move  a  cockpit 
control  is  not  necessarily  represented 
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properly  in  the  time-compression  mode  of  simulation.  This  is 
especially  true  if  the  motor-skill  learning  (in  real  time)  has 
progressed  to  the  automatic  phase.  Thus,  there  is  some  concern 
about  the  physical  de-training  aspects  of  the  time-compression 
simulator . 

Pilot  Interviews 

In  an  attempt  to  explore,  at  least  subjectively,  some  of  the 
critical  aspects  of  time-compressed  simulation,  four  F-15  RPRV 
test  pilots  were  interviewed  as  a  group.  Each  of  these  pilots  was 
a  highly  experienced  test  pilot  with  several  thousand  hours  of 
flying  time  in  a  wide  variety  of  test  aircraft  in  addition  to  his 
"arm  chair"  flights  of  the  F-15  RPRV.  The  questions  asked,  and  a 
composite  summary  of  their  answers  follows: 

1.  Q.  Do  you  feel  that  the  realism  of  the  1.4  time- 
compressed  simulation  is  truly  a  psychological  effect? 

A.  In  general,  yes.  One  pilot  felt  that  it  was  mostly 
psychological  but  that  there  were  still  differences  between 
simulation  and  flight  due  to  flight  discrepancies  (i.e.  flight 
events  which  are  not  exactly  simulated) . 

2.  Q.  How  critical  is  the  time-compression  factor  of  1.4? 

A.  1.4  was  the  only  factor  available  to  the  F-15  RPRV 

simulation  but  all  were  satisfied  that  it  was  a  good  factor  for 
this  task. 

3.  Q.  What  were  the  environmental  differences  between 
simulator  training  and  actual  flights? 
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A.  (1)  The  simulator  had  a  map  display  which  the  pilot 
used  directly  to  perform  navigation.  During  actual  flights 
navigation  was  done  by  the  mission  controller  and  directions  were 
verbally  passed  to  the  pilot  (such  as  "turn  right  10°") . 

(2)  The  room  and  cockpit  lighting  were  different 
during  the  actual  flights. 

(3)  There  was  isolation  from  outside  distractions  on 

flight  day. 

(4)  There  was  more  copilot-type  help  during  actual 
flights,  such  as  altitude  calls  and  other  verbal  reminders. 

4.  Q.  Is  there  a  learning  curve  associated  with  the  1.4 
time-compression  factor? 

A.  This  was  difficult  to  determine  since  the  philosophy 
was  to  increase  the  pilot  work  load  on  successive  flights  in 
order  to  maximize  data  return.  At  least  one  pilot  used  the  1.4 
time-compression  runs  as  a  means  of  establishing  a  balance 
between  the  magnitude  and  complexity  of  the  requested  maneuvers, 
and  his  training  readiness  to  accomplish  them. 

With  this  philosophy  in  mind,  the  1.4  factor  appeared  to  be 
valid  for  all  flights. 

5.  Q.  How  repeatable  is  the  psychological  effect  of  the 
1.4  time-compression? 

A.  (See  answer  to  4) . 

6.  Q.  Are  you  bothered  by  incorrect  physical  response 
times  when  simulating  in  the  time-compressed  mode? 
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A.  Physical  differences,  such  as  the  time  required  to 
reach  and  actuate  a  switch,  were  apparent  to  the  pilots,  but  all 
agreed  that  the  effects  were  minor  compared  to  the  training 
benefit  derived  from  the  time-compressed  mode. 

7.  Q.  What  was  the  ratio  of  real  time  training  runs  versus 
time-compressed  training  runs  before  a  typical  flight? 

A.  Approximately  70%  of  the  training  runs  were  in  real 
time  (early  learning  of  the  flight  plan,  primarily)  and  30%  were 
in  1.4  time-compression  (the  last  day  or  sO  before  flight)  . 

8.  Q.  Do  you  feel  more  relaxed  and  less  rushed  during 
flight  if  you  have  had  the  time-compression  training? 

A.  More  relaxed?  — No. 

Less  rushed  and  more  confident?  —  Definitely  Yes. 

(Nearly  all  RPRV  flights  used  the  1.4  time-compression 
training  so  the  answer  relates  more  to  overall  flying 
experience) . 

9.  Q.  What  biomedical  measurements  were  being  taken  during 
RPRV  flights  and  what  were  the  results? 

A.  EKG  was  being  recorded  on  both  the  pilot  and  the 
mission  controller.  Work  load,  as  evaluated  qualitatively  by 
pilots  and  quantitatively  by  heart  rate  measurement,  were 
comparable  to  or  higher  than,  the  work  load  experienced  by  the 
same  pilots  when  they  were  actually  flying  the  rocket  powered 
lifting  bodies. 

One  highly  experienced  test  pilot  who  has  made  many  flights 
in  four  different  rocket  powered  lifting  bodies  felt  that  the 
pilot  work  load  during  his  first  RPRV  landing  was  the 
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highest  he  had  ever  experienced  and  a  maximum  for  him  personally. 
EKG  data  confirmed  that  his  heart  rate  was  as  high  during  the 
RPRV  landing  as  during  any  of  his  actual  landings  in  unpowered 
lifting  bodies. 

10.  Q.  What  do  you  see  as  advantages  of  time-compression 
simulation? 

A.  All  pilots  felt  that  there  was  obvious  application  to 
any  high  work  load  piloting  task  such  as  emergency  training, 
instrument  landing  approaches,  and  many  others.  All  of  the  pilots 
were  highly  enthusiastic  about  its  value,  at  least  in  this 
particular  application.  All  felt  that  it  forced  the  pilot  to 
concentrate  on  the  important  aspects  of  the  simulation. 

11.  Q.  What  do  you  see  as  disadvantages?  . 

A.  There  are  minor  effects  due  to  physical  differences 
(question  3)  which  could  cause  some  training  distortion.  Pilots 
reiterated  that  time-compression  should  not  be  used  for  all 
training.  Some  balance  of  training  for  motor  skills  (real  time) 
and  mental  skills  (1.4  time— compression)  is  required. 

12.  Q.  Comment  on  the  following  possible  side  benefits 

(A)  Evaluating  trainee  readiness  (go-no-go) 

(B)  Evaluating  task  planning  effectiveness 

(C)  Screening  or  selecting  pilot  candidates 

A.  All  of  the  pilots  felt  that  all  three  items  were  quite 
valid  potential  applications  of  time-compression  simulation.  One 
pilot  had  already  been  consciously  applying  (A)  and  (B)  to  his 
own  training. 
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GAINING  ACCEPTANCE 


Documentation  of  the  application  of  time-compressed 
simulator  training  to  the  F-15  RPRV  program  has  not  yet  received 
wide  dissemination.  On  at  least  one  occasion  the  subject  was 
mentioned  to  a  simulator  training  expert  employed  by  a  major 
airline.  When  it  was  suggested  that  they  might  try  speeding  up 
their  simulators  his  reaction  was  highly  negative.  "We're  trying 
to  teach  the  crews  to  slow  down  their  responses  in  an  emergency, 
not  speed  up .  We  have  enough  trouble  with  pilots  reacting  too 
fast  and  doing  the  wrong  thing." 

It  is  anticipated  that  this  type  of  response  may  be  typical 
for  someone  who  has  not  totally  analyzed  the  problem.  If  the 
motor  skills  for  handling  an  emergency  situation  are  learned  only 
in  a  real  time  simulation,  then  the  psychological  effect  of 
stress  during  a  real  occurrence  will  cause  the  pilot  to  feel 
behind  mentally  and  therefore  rushed  in  his  activity  (thus  more 
likely  to  make  errors) .  If,  after  learning  the  motor  skills  in 
real  time,  the  pilot  had  been  exposed  to  a  time-compression 
simulation  with  instructions  that  "this  is  how  it  will  feel", 
then  the  real  occurrence  would  have  presented  him  with  a  more 
familiar  situation  which  he  could  handle  more  confidently  (thus 
less  likely  to  make  errors) . 

Quantitative  statistical  confirmation  of  benefits  of  time- 
compression  simulation  may  take  many  years  to  accrue.  Selling  the 
concept  of  time-compression  simulation  may. 
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therefore,  have  some  initial  hurdles  to  overcome.  A  simple 
demonstration  is  probably  the  easiest  and  most  convincing  way  of 
gaining  acceptance.  Generalized  computer  software  modifications 
which  would  allow  time-compression  simulation  to  be  done  on 
almost  any  simulator  should  first  be  developed.  A  high  work  load 
task  which  can  be  quickly  tried  on  a  simulator  and  verified  in 
the  real  world  (rapid  feedback)  should  then  be  demonstrated  to  a 
potential  user  (an  airliner  landing  simulation  for  example) .  If 
time-compression  simulation  is  truly  applicable  to  the  task,  the 
trainee  will  recognize  the  benefits  almost  immediately  and 
hopefully  will  be  able  to  convince  his  own  management  of  its 
value.  Additional  laboratory  work  to  quantify  time-compression 
factors  for  various  different  tasks  or  work  loads  is  essential  to 
any  wider  application  of  time-compression  simulation  to 
situations  different  from  the  one  discussed  in  this  paper.  The 
concept  could  be  improperly  applied  if  rapid  and  effective 
trainee  feedback  were  not  heavily  emphasized. 


SUMMARY 

Biomedical  measurements  of  test  pilots  flying  Remotely 
Piloted  Vehicles  (RPRV's),  when  correlated  with  past  data  taken 
in  flight,  has  strongly  indicated  that  the  stress  levels  and 
physical  and  mental  states  of  test  pilots  are  primarily 
influenced  by  the  strong  sense  of  responsibility  and  resulting 
anxiety,  rather  than  by  fear  of  personal  harm  or  direct 
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physiological  stress.  It  appears  that  this  mental  state  can  be 
approximately  simulated  under  non— stressful  conditions  by 
increasing  the  simulated  rate  of  time  passage  (time-compression 
simulation).  A  time-compression  factor  of  1.4,  as  used 
successfully  on  the  NASA  F-15  RPRV  program,  appears  to  be 
appropriate  for  relatively  short,  high  work  load,  piloting  tasks 
(Simulator  operates  1.4  times  faster  than  real  time). 

The  concept  of  time-compression  simulation  appears  to  be 
applicable  to  a  wide  range  of  simulator  training  situations  where 
stressful,  operator— in— the— loop  tasks  are  being  simulated  under 
non-stressful  conditions. 

Further  testing  under  controlled  conditions  should  be 
performed  to  establish  a  relationship  between  the  time- 
compression  factor  and  either  work  load,  or  a  direct 
physiological  measure,  such  as  heart  rate. 
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Appendix  C 

F-16  Initial  Conditions  Used  In  Experiments  1  and  3. 


Bandits.  The  bandits  are  all  MiG-23 s  assumed  to  be  in  air-to-ground  roles  as  bombers.  All 
contacts  are  considered  hostile  in  the  Radar  Skills  Phase.  A  visual  identification  is  required 
(within  6,000  ft  +  30°  of  stem  aspect)  for  the  Intercept  Phase.  Bandits  simulate  a  limited  forward 
looking  radar  capability  and  limited  threat  ground-controlled  intercept  coverage. 

F-16  Configuration 
AIM-9LIM  on  stations  1  and  9 
AIM- 120  on  stations  2, 3,  7,  and  8 
Stations  4,  5,  and  6  clean  (no  external  tanks) 

Avionics  Setup 

Master  Arm  -  Master  Arm  position 
Right  MFD  -  SMS 

Inventory  as  described  above 
AIM-120  in  SLAVE 
AIM-9s  cooled,  SLAVE,  BP,  SPOT 
Left  MFD  -  Radar 

FCR  Control  Page  (items  of  significance) 

MTR  LO 

ALT  TRK  -  OFF 

TGTH1S-3 

Hands-on  Controls  -  ANT  ELEV  Knob  in  detent 
Dogfight  Switch 

Center  Position  -  A-A  Master  Mode  selected: 

Radar  SMS 

TWS  -  MAN  AAM  operating  mode 

8  0  NM  range  AIM- 1 20  selected 

3  B 

A6 

Cursors  centered  on  MFD 
FOV-NORM 

MSL  OVRD  Position:  Radar  SMS 

RWS  MSL  operating  mode 

80  NM  AIM- 120  selected 

4B 
-  A6 

Cursors  centered  on  MFD 

DGFT  Position:  Radar  SMS 

ACM  DGFT  operating  mode 

EEGS 

AIM-9 
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Appendix  D 

Experimenter’s  Rating  Forms  for  Radar  Skills  and  Intercept  Training 
Radar  skills  experiment  1. 
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Radar  skills  experiment  3. 


PILOTS  NAME 


PILOT  NUMBER 


TRAINING  START  TIME: 


RADAR  SKILLS  TRAINING 


INSTRUCTOR 


SCENARIO  NUMBER 


DATE 


TRAINING  STOP  TIME: 


MISSION  TASKS: 


SEARCH  AIRSPACE _ 

SAMPLE  CONTACTS _ 

SORT  FORMATION  AND  MONITOR  ACTION _ 

TARGET  HIGHEST  THREAT  (SHOOT  IN  PARAMETERS) _ 

DEBRIEF: _ _ 

DESCRIBE  INITIAL  PICTURE _ 

DESCRIBE  INITIAL  ACTION _ 

DESCRIBE  PICTURE  AFTER  ACTIONS _ 

DESCRIBE  FACTORS  USED  TO  DETERMINE  HIGHEST  THREAT 


COLUMN  TOTALS: 
TOTAL  SCORE: 


J _ L 


GRADING  CRITERIA: 

Grade  3  -  Performance  is  correct 

Grade  2  -  Performance  is  essentially  correct,  but  late 

Grade  1  -  Performance  only  partially  correct,  errors  of  omission  or  commission 
Grade  0  -  Performance  totally  incorrect 

Notes: 


Intercept  task  experiment  1. 


INTERCEPT  TRAINING 

PILOTS  NAME 

INSTRUCTOR 

PILOT  NUMBER  SCENARIO  NUMBER  DATE 

TRAINING  START  TIME: 

TRAINING  STOP  TIME: 

MISSION  TASKS: 

GRADE 

0 

1 

2 

3 

SEARCH  AIRSPACE 

SAMPLE  CONTACTS 

SORT  FORMATION  AND  MONITOR  ACTION 

TARGET  TRAILING  THREAT(S) 

OBTAIN  PROPER  OFFSET 

CONVERT  TO  STERN  (WITHIN  30  DEG.  AND  6000  FT) 

SHOOT  IN  PARAMETERS 

1 

DEBRIEF: 

555 

23 

DESCRIBE  INITIAL  PICTURE 

DESCRIBE  ACTION 

DESCRIBE  PICTURE  AT  20NM 

DESCRIBE  FACTORS  USED  TO  DETERMINE  HIGHEST  THREAT 

COLUMN  TOTALS 

TOTAL  SCORE 

0 

A  total  score  of  8  is  required  for  proficiency.  Base  this  score  on  debrief  questions  only! 


GRADING  CRITERIA: 

Grade  3  -  Performance  is  correct 

Grade  2  -  Performance  is  essentially  correct,  but  late 

Grade  1  -  Performance  only  partially  correct,  errors  of  omission  or  commission 
Grade  0  -  Performance  totally  incorrect 


Notes: 


Appendix  E. 

TLX  Rating  Form 

RADAR  SKILLS  TRAINING  TLX  Subject  # _ 

Below  is  a  list  of  descriptions  concerning  various  aspects  of  the  scenarios  you  just  completed. 
You  will  be  asked  to  make  ratings  and  comparisons  of  each  of  these.  Please  review  each  of  the 
descriptions  and  keep  them  in  mind  as  you  fill  out  this  document. 

PHYSICAL  DEMAND  (PD)  Concerns  the  amount  of  physical  activity  that  was  required.  For 
example,  the  amount  of  button  pressing,  reaching,  control  adjustments,  etc.  that  was  involved 
during  the  task. 

TIME  PRESSURE  (TP)  Concerns  how  much  time  pressure  you  felt  due  to  the  rate  or  pace  at 
which  the  task  elements  occurred  (for  example,  slow  and  easy  or  rapid  and  frantic). 

INTERPRETING  RADAR  (IR)  Concerns  how  much  of  your  mental  effort  you  delegated  to 
interpreting  elements  of  your  radar,  including  items  such  as  finding  and  sorting  the  bogies, 
determining  the  formation  of  the  bogies,  and  tactical  planning 

SWITCHOLOGY  (SW)  Concerns  how  much  of  your  total  effort  (both  mental  and  physical)  was 
directly  related  to  performing  aspects  of  switchology  for  radar  and  weapons. 

FLYING  PLANE  (FP)  Concerns  how  much  of  your  total  effort  (both  mental  and  physical)  was 
directly  related  to  aspects  of  flying  the  aircraft  (for  example,  stick  and  throttle  controls). 

FRUSTRATION  LEVEL  (FL)  Concerns  how  discouraged,  irritated,  or  stressed  you  were  while 
performing  the  scenarios. 

INSTRUCTIONS:  Please  select  the  member  of  each  pair  that  contributed  more  heavily  to  your 
workload  during  the  scenarios.  That  is,  which  member  of  each  pair  affected  your  performance 
more?  Please  base  your  responses  on  all  of  the  radar  skills  training  you  just  received,  not  just 
one  particular  scenario.  It  is  important  to  complete  each  comparison,  even  if  members  of  one 
pair  seemed  to  have  affected  your  performance  equally. 


PD /TP 
PD  /  IR 
PD/SW 
PD/FP 
PD/FL 


TP/IR 
TP/SW 
TP/FP 
TP/FL 
IR  /  SW 


IR/FP 

IR/FL 

SW/FP 

SW/FL 

FP/FL 
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Now  please  rate  individually,  on  a  scale  from  1  to  10,  the  magnitude  of  the  contribution  of  each 
factor  on  your  performance  during  the  scenarios  you  just  completed. 


EFFECT  ON  YOUR  PERFORMANCE 
TYPE  LOW  HIGH 


Physical  Demand 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Time  Pressure 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Interpreting  Radar  1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Switchology 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Flying  Plane 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Frustration  Level 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

i 
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